Adult and Community Learning Services (ACLS)

Measurable Skills Gain (MSG)

The state MSG performance standard includes the following outcomes: Educational Functioning Level (EFL) completion, High School Equivalency (HSE) credential attainment, and Postsecondary Education or Training (PSE/T) enrollment. Only outcomes achieved during the fiscal year will be included in MSG. In addition, postsecondary enrollment must start after a student exits from adult education.

In FY19 and annually thereafter, programs will be assigned separate ABE and ESOL targets (programs that provide both services will get two targets) using a formula that incorporates (i.e., is weighted on):

  1. The number of students who pre-test into each of the National Reporting System (NRS) levels that a program serves, and
  2. The degree of difficulty in achieving an MSG outcome at each level1 (based on the federal targets assigned to MA and adjusted to include the possibility of multiple outcomes).

Program performance will be the program's actual MSG relative to the target. For example, an ESOL program with a target of 45% and an actual MSG of 50% will have exceeded its target by 11%; their performance would be considered 111% relative to target. Another program with a target of 55% and actual MSG of 50% will be 91% of target.

Note: Programs will get one credit (i.e., 1.0) for the first MSG outcome earned and will get a partial credit (i.e., 0.1) for each additional outcome earned beyond the initial. If students in a program achieve an initial outcome and two additional outcomes, then the program's credit for that student is 1.2. Only outcomes achieved during the fiscal year will be included in MSG.

MSG outcomes, both initial and additional partial, can be any of the following:

  • Educational Functioning Level (EFL) completion
  • High School Equivalency (HSE) credential attainment
  • Postsecondary Education or Training (PSE/T) enrollment after exit

Here are some examples of how initial and partial credit is distributed for student outcomes:

  • Two EFL completions (i.e., advancement in two NRS levels) = 1.1
  • EFL completion in two subject areas (e.g., reading, math) = 1.1
  • High School Equivalency (HSE) credential attainment and two EFL completions = 1.2
  • Two EFL completions in two subject areas plus enrollment in Postsecondary Education or Training (PSE/T) after exit = 1.4

MSG Targets Assigned by ACLS

ACLS will post preliminary ABE and ESOL program targets after summer and fall pre-tests have been entered into LACES. Targets may change slightly over the year since one of two factors in an MSG target is the number of students who pre-test into each of the program's NRS levels and pre-testing will continue throughout the year.

Programs may calculate their targets using the FY19 Target Calculator. Download the Excel calculator and enter the number of students who pre-test at each NRS level.

Download Excel Document
FY2019 Target Calculator

The MSG target percentages are based on federal targets assigned to Massachusetts and adjusted upward because of the opportunity for multiple outcomes for individual students. OCTAE has raised Massachusetts' target each year since the start of WIOA. As OCTAE raises Massachusetts' targets, ACLS will raise program targets.

The preliminary FY19 targets posted below were based on LACES data as of February 22, 2019. ACLS intends to assign programs their preliminary ABE and ESOL targets in late October of each year after summer and fall test data are entered.

Download Excel Document
FY2019 Preliminary MSG Program Targets

FY2018 MSG Program Performance

The Excel document, FY18 MSG Performance, has 17 sortable columns:

  1. Program
  2. Number of ABE students
  3. Number of ABE students who completed at least one EFL
  4. Number of HSEs
  5. Number of ABE students who enrolled in PSE/T
  6. Number of PoPs
  7. Program MSG ABE target
  8. Program actual ABE MSG (outcomes/PoPs)
  9. Percent of ABE target met (actual MSG/target)
  10. Number of ESOL students
  11. Number of ABE students who completed at least one EFL
  12. Number of HSEs
  13. Number of ESOL students enrolled in PSE/T
  14. Number of PoPs
  15. Program MSG ESOL target
  16. Program actual MSG (outcomes/PoPs)
  17. Percent of ESOL target met (actual/target)

Assigned targets (columns G and O) and percentage of targets met (I and Q) are rounded to the whole number. Actual MSG (H and P) are rounded to the tenth percentage2.

Download Excel Document
FY2018 MSG Performance

It is important to note that MSG is based on PoPs rather than students and that most actual MSG is a combination of initial full credit outcomes and additional partial credit outcomes. The FY2018 MSG Performance spreadsheet includes student numbers and outcomes that will provide insight into a program's performance; it does not show whether the outcomes are initial outcomes (1.0 credit) or partial (0.1 credit when a student achieves an additional outcome). Consider the example of Wyvonne's Wicked Good Adult Education Program:

ProjectStudentsEFL CompletersHSEsPSE/TsPoPsMSG TargetMSG %% of Target Met
WWGAEP188212031%23.5%76%

Why is the MSG only 23.5% and not 55% when there are eleven outcomes (11 ÷ 20)? The worksheet does not show that all the outcomes were achieved by only four students (four initial MSG credits and seven partial credits). The actual performance is 23.5% (4.7 ÷ 20). ACLS has not developed a way to provide this level of detail about program outcomes. Directors can request detailed program performance information by contacting Brian Newquist, ACLS Data Analyst.

MSG and Grantee Evaluation

MSG is one component of grantee performance evaluation. The others are Program Quality Reviews, site visits, desk (data) monitoring, and risk analysis. MSG is not designed to rank grant recipients.


1 For reporting and setting targets, all students are placed into one of the twelve NRS levels. This is their initial entry level. The initial entry level is determined by the first assessment in the fiscal year, regardless of the number of PoPs. For detailed information about initial entry level refer to the Massachusetts Adult Education Assessment Policies Download Word Document.

2 The FY18 MSG worksheet that was attached to the January 23, 2019 memo differs slightly from this worksheet because the rounding methodology was not applied.

 
Contact Us

Massachusetts Department of Elementary and Secondary Education
75 Pleasant Street, Malden, MA 02148-4906

Voice: (781) 338-3000
TTY: (800) 439-2370

Directions

Disclaimer: A reference in this website to any specific commercial products, processes, or services, or the use of any trade, firm, or corporation name is for the information and convenience of the public and does not constitute endorsement or recommendation by the Massachusetts Department of Elementary and Secondary Education.