English Language Learners
Guide to Understanding the 2008 Annual Measurable Achievement Objectives (AMAOs) Reports
This document describes the methodology used to establish Annual Measurable Achievement Objectives (AMAOs) based on Massachusetts English Proficiency Assessment (MEPA) and Adequate Yearly Progress (AYP) data from the 2007-08 school year.
Title III of the No Child Left Behind Act (NCLB) requires that limited English proficient (LEP) students be assessed annually to measure both their progress toward English language proficiency and their attainment of English language proficiency. In 2007-08, LEP students in grades 3-12 took both the Massachusetts English Language Assessment-Oral (MELA-O), and the reading and writing components of MEPA tests (MEPA-R/W). MEPA for students in grades 3-12 comprises four different grade-span tests: Grades 3-4; Grades 5-6; Grades 7-8; Grades 9-12.
MEPA is administered in both fall and spring of each academic year. All grade 3 LEP students, as well as LEP students newly transferring into Massachusetts schools in grades 3-12, must participate in the fall MEPA tests in order to establish a baseline score. All LEP students in grades 3-12 must participate in spring MEPA tests. In 2007-08, LEP students in grades K-2 were assessed using the MELA-O for speaking and listening, and the IDEA Language Proficiency Test (IPT) for reading and writing.
MEPA results are reported in four performance levels: Beginning; Early Intermediate; Intermediate; Transitioning. Additionally, each of the first three performance levels on the MEPA has been divided into two performance level steps. For students in grades K-2, the Department has divided performance levels results in both the MELA-O and IPT into five distinct performance level steps.
The Massachusetts Department of Elementary and Secondary Education is required by Title III of the No Child Left Behind Act (NCLB) to establish annual achievement objectives (AMAOs) for the English language acquisition and academic achievement of LEP students. AMAOs are produced for students in grades K-12, and comprise three categories: (1) progress toward acquiring English language proficiency; (2) attainment of English language proficiency; and (3) Adequate Yearly Progress (AYP) of the LEP subgroup in both English Language Arts and in mathematics.
The Department generates AMAO reports for the State overall and for all districts receiving federal Title III funds.
2008 Progress Target
Progress determinations are based on LEP students who participated in all portions of the MEPA examination in spring 2008 and who had a baseline performance level, either from spring 2007 or fall 2007.
For students whose baseline MEPA score is from the same grade-span test as in spring 2008, the Department has defined progress toward acquiring English language proficiency as advancing by two or more performance level steps on the MEPA. For students whose baseline MEPA score is from a different grade-span test than in spring 2008, progress is defined as advancing one or more steps. Students with baseline MEPA scores at the Transitioning level who again scored at Transitioning in spring 2008 are also considered to have made progress.
Progress determinations are based on LEP students who participated in all portions of the MELA-O and IPT assessments in spring 2008 and who had a baseline performance level from spring 2007. Students who were in kindergarten for the first time in 2008 were included in the progress determinations if they participated in all portions of the MELA-O and IPT assessments in spring 2008.
Of the three IPT performance levels, each of the two lowest levels are divided into two steps, based on raw scores. The highest performance level is undivided, and is assigned as Step 5. The five scores associated with the MELA-O Listening test are translated directly into steps. For the MELA O Speaking assessment, the steps are assigned as follows: 0-4 points = Step 1, 5-8 = Step 2, 9-12 = Step 3, 13-16 = Step 4, 17-20 = Step 5.
The Department has defined progress toward acquiring English language proficiency for K-2 students as increasing one or more performance level steps from spring 2007 to spring 2008 or, due to the substantial differences in cognitive and language demand between the kindergarten, grade 1, and grade 2 tests, maintaining the same performance as the prior year, depending on grade level. To be considered as making overall progress, a student must make progress in three of the four assessments (MELA-O Speaking, MELA-O Listening, IPT Reading, IPT Writing).
The state target for 2008 is that sixty-two percent (62%) of LEP students will make progress toward acquiring English proficiency. Overall district performance is measured against this target.
2008 Attainment Target
Attainment of English language proficiency is defined as scoring at the Transitioning level on MEPA. State data and research indicate that there are two distinct factors that most closely correlate with the likelihood that a student will reach the Transitioning level on the MEPA test by the end of a given year: (a) the number of years a student has been in U.S. schools; and (b) his or her English language proficiency at the start of that year (the student’s "baseline" performance). Only students with complete MEPA results from spring 2008 and complete baseline results are included in these calculations.
Analysis of statewide data shows that the characteristics of the LEP student population, and therefore the two factors identified above, vary widely from district to district. Based on this distribution, the Department used statewide data to set individual 2008 attainment targets for each district based on the two factors.
A matrix displaying the 2008 percentage targets for attainment, by years in U.S. and baseline performance, is below.
|Years in US||Performance Level Step (2007 Baseline)|
For example, our target is that 20 percent of students in their 2nd year of U.S. schooling who scored at Step 3 in 2007 to reach Transitioning in 2008. Additionally, our target is that 65 percent of students in their 3rd year of U.S. schooling who scored at Step 6 in 2008 to reach Transitioning in 2008.
Because the composition of each district’s LEP population is different, each district has a different target for its Year 2, Year 3, Year 4 and Year 5+ students. Individual district targets are established by multiplying the number of district students at each baseline performance level step by the state targets for attainment in the matrix above and combining the individual year in U.S. targets. District targets are reset each year depending on the composition of the LEP students served. LEP students in their 1st year of U.S. schooling are not included in the attainment target calculation.
Through this approach, a district whose Year 2 students were by-and-large very close to Transitioning at the beginning of the year will have a much higher target than a district whose majority of Year 2 students was still in the Beginning or Early Intermediate levels. A district must meet its overall target and at least half of its individual year targets to receive a positive AMAO determination for Attainment.
Attainment of English language proficiency for students in grades K-2 is defined as reaching Step 4 or higher in three out of the four K-2 assessments (MELA-O Speaking, MELA-O Listening, IPT Reading, IPT Writing) administered in Spring 2008.
Similar to the grade 3-12 MEPA attainment targets described above, the Department established district attainment targets for grades K-2 based on the performance distribution of students the district served. Because there are four different K-2 assessments, however, the attainment targets are based on the number of assessments in which each student scored at the proficient level or higher. The district target for each kindergarten student was set at 40 percent.
|Grade in 2008||Number of Assessments Student was Proficient in During 2007|
|1 or 2||20||30||40||60||80|
Individual targets are summed and then divided by the number of students in the district.
Thus a district serving 100 kindergartners and 100 1st graders who were not proficient on any of the tests the prior year would have a K-2 target of 30 percent (100 x 4000 = 40; 100 x 20 = 2000; 6000 ÷ 200 = 30).
AYP in English language Arts and Mathematics for LEP Subgroup
To receive an affirmative Adequate Yearly Progress (AYP) determination, a district must meet: 1) the 95% MCAS student participation requirement; 2) the 92% average attendance target or 1% attendance improvement; and either (3) the State's 2008 performance target for each subject or (4) the district's own 2008 improvement target. The 2008 performance targets are a Composite Performance Index (CPI) of 85.4 points for ELA and 76.5 for mathematics. The amount of CPI increase a district is expected to achieve during a particular rating cycle is based on the gap between that district's baseline CPI and a CPI of 100 (the year 2014 performance target for all Massachusetts schools and districts).
AYP determinations used in 2008 AMAO reports are based on results for LEP students in all grades combined (English language arts and mathematics: grades 3-8 & 10). More information on how AYP is determined is available at: http://www.doe.mass.edu/sda/ayp/2008/.