Adult and Community Learning Services (ACLS)

FY2019 MSG Performance Report Corrected Version

September 6, 2019

Dear Program Director,

I have attached the corrected FY19 MSG report . We apologize for the confusion and the delay and are grateful to those of you who caught the errors and notified us. The information below is the same as in the original letter.

The state MSG performance standard includes the following outcomes: Educational Functioning Level (EFL) completion, High School Equivalency (HSE) credential attainment, and Postsecondary Education or Training (PSE/T) enrollment. Only outcomes achieved during the fiscal year are included in MSG. In addition, postsecondary enrollment must start after a student exits from adult education.

Programs are assigned separate ABE and ESOL targets (programs that provide both services get two targets) using a formula that incorporates (i.e., is weighted on):

1. The number of students who pre-test into each of the National Reporting System (NRS) levels (pages 21 through 32) that a program serves, and
2. The degree of difficulty in achieving an MSG outcome at each level (based on the federal targets assigned to MA and adjusted to include the possibility of multiple outcomes).

Program performance is the program's actual MSG relative to the target. For example, an ESOL program with a target of 45% and an actual MSG of 50% has exceeded its target by 11%; their performance is considered 111% relative to target. Another program with a target of 55% and actual MSG of 50% is 91% of target.

Note: Programs receive one credit (i.e., 1.0) for the first MSG outcome earned and get a partial credit (i.e., 0.1) for each additional outcome earned beyond the initial. If students in a program achieve an initial outcome and two additional outcomes, then the program's credit for that student is 1.2. Only outcomes achieved during the fiscal year are included in MSG.

MSG outcomes, both initial and additional partial, can be any of the following:

• Educational Functioning Level (EFL) completion
• High School Equivalency (HSE) credential attainment
• Postsecondary Education or Training (PSE/T) enrollment after exit

Here are some examples of how initial and partial credit is distributed for student outcomes:

• Two EFL completions (i.e., advancement in two NRS levels) = 1.1
• EFL completion in two subject areas (e.g., reading, math) = 1.1
• High School Equivalency (HSE) credential attainment and two EFL completions = 1.2
• Two EFL completions in two subject areas plus enrollment in Postsecondary Education or Training (PSE/T) after exit = 1.4

The attached Excel document, FY19 MSG Performance, has 17 sortable columns:

1. Program
2. Number of ABE students
3. Number of ABE stud
   ents who completed at least one EFL
4. Number of HSEs
5. Number of ABE students who enrolled in PSE/T
6. Number of PoPs
7. Program MSG ABE target
8. Program actual ABE MSG (outcomes/PoPs)
9. Percent of ABE target met (actual MSG/target)
10. Number of ESOL students
11. Number of ABE students who completed at least one EFL
12. Number of HSEs
13. Number of ESOL students enrolled in PSE/T
14. Number of PoPs
15. Program MSG ESOL target
16. Program actual MSG (outcomes/PoPs)
17. Percent of ESOL target met (actual/target)

Assigned targets (columns G and O) and percentage of targets met (I and Q) are rounded to the whole percent number. Actual MSG (H and P) are rounded to the tenth of a percentage point.

It is important to note that MSG is based on PoPs rather than students and that most actual MSG is a combination of initial full credit outcomes and additional partial credit outcomes. The FY2019 MSG Performance spreadsheet includes student numbers and outcomes that will provide insight into a program's performance; it does not show whether the outcomes are initial outcomes (1.0 credit) or partial (0.1 credit when a student achieves an additional outcome). Consider the example of Wyvonne's Wicked Good Adult Education Program:

Why is the MSG only 23.5% and not 55% when there are eleven outcomes (11 ÷ 20)? The worksheet does not show that all the outcomes were achieved by only four students (four initial MSG credits and seven partial credits). The actual performance is 23.5% (4.7 ÷ 20). ACLS has not developed a way to provide this level of detail about program outcomes. Directors are encouraged to analyze their LACES data. LiteracyPro Systems will provide training and is developing supporting documents and videos to help you. You may also contact Brian Newquist, ACLS Data Analyst, and request detailed program performance information. He will be able to respond after ACLS submits our federal reports in early October.

Please note that MSG is one component of grantee performance evaluation. The others are Program Quality Reviews, site visits, desk (data) monitoring, and risk analysis. MSG is not designed to rank grant recipients.

Sincerely,

Wyvonne Stevens-Carter
Adult Education State Director