The Massachusetts Board of Elementary and Secondary Education

Methodology for identifying Level 3 Schools (bottom 20% of schools)

Ch. 69 M.G.L. (1)(J)(a) "Schools that score in the lowest 20 per cent statewide among schools serving common grade levels on a single measure developed by the department that takes into account student performance data and, beginning on July 1, 2011, improvement in student academic performance, shall be deemed eligible for designation as underperforming or chronically underperforming. Not more than 4 per cent of the total number of public schools may be designated as underperforming or chronically underperforming at any given time."

N.B. The legislation in this context does not include charter schools.

1. Filter out all charter schools (in 2009-2010, that number was 1,831-63 = 1,768).

2. Find 20% of that number (do this before you do any more filtering, as the law does not say "the lowest 20 percent of those schools with relevant data"). This is your target number (354).

3. Filter out EECs.

4. Use frequencies to determine the remaining 6 different school types (ES, ESMS, MS, MSHS, HS, K-12). We interpreted "schools serving common grade levels" as these 6 types, because otherwise it's an impossible task with all of the various grade configurations in this state.
   Of non-charter, non-EEC schools, in 2009-2010:

   • 54.9% were ES
   • 6.5% were ESMS
   • 17.4% were MS
   • 3.5% were MSHS
   • 17.1% were HS
   • .6% were K-12

5. Multiply those proportions by the target number (1,768), and those 6 numbers become your target numbers for those 6 school types.

   • 194 ES
   • 23 ESMS
   • 62 MS
   • 12 MSHS
   • 60 HS
   • 3 K - 12

6. Filter out small schools and schools without performance data over the past 4 years (%w/f, cpi, both subjects)

7. Run the level 4 performance syntax, and rank. Take the lowest X, where X = the target number for that particular school type.

8. For this year, we had to include some small schools (8) because they were eligible for SIG (tier X) because of the special requirements for SIG. They could not be SIG tier X and anything less than Level 3. So, we included those, and replaced the highest performing school in the school type list.

The level 3 school determination is based on this language contained in the Education Reform Act of 2010. Essentially, the law states that a school scoring in the lowest 20 percent statewide should be considered eligible for designation as an underperforming, or level 4, school. Of the lowest performing 20 percent of schools (354 schools), 35 were designated as level 4 schools in 2010. The rest of those 20 percent (319 schools) were designated as level 3 schools.

In school year 2009-2010, the Commonwealth had 1,768 schools in operation (excluding charter schools). The lowest 20 percent, therefore, would consist of 354 schools. As the law states, these schools must serve common grade levels, because it would not be valid to compare an elementary school's test results to those of a high school. Thus, to facilitate valid performance comparisons, we placed schools into six common grade categories:

Elementary Schools:	Usually grades K-5 or K-6
Elementary/Middle Schools:	Usually grades K-8
Middle Schools:	Usually grades 6-8 or 7-8
Middle/High Schools:	Usually grades 7-12
High Schools:	Usually grades 9-12
K - 12 Schools:	Grades K-12

We then compared the schools within each of these categories to determine the lowest 20% in terms of performance using CPI and the percentage of students scoring Warning/Failing on the MCAS ELA and mathematics tests over the past four years. We ensured that the schools selected were proportionate to the number of schools in each common grade category, producing 194 elementary schools, 23 elementary/middle schools, 62 middle schools, 12 middle/high schools, 60 high schools, and three K - 12 schools. The 35 level 4 schools were selected from among these schools, and the 319 remaining schools were designated as level 3 schools.