Mass.gov
Massachusetts Department of Elementary and Secondary Education
Go to Selected Program Area
Massachusetts State Seal
Students & Families Educators & Administrators Teaching, Learning & Testing Data & Accountability Finance & Funding About the Department Education Board  
>
>
>
>
>
>
>

The Massachusetts Board of Elementary and Secondary Education

Weighting of Achievement and Growth in Accountability Calculations

To:
Members of the Board of Elementary and Secondary Education
From:
Mitchell D. Chester, Ed.D., Commissioner
Date:
June 20, 2014

line

At the meeting of the Board of Elementary and Secondary Education on June 24, 2014, we will continue our discussion regarding the weighting of achievement and growth indicators in our school accountability calculations and in the calculations used to identify the lowest performing districts for purposes of the charter cap lift. We discussed this topic at length at the Board's special meeting on April 28, at its regular meeting on April 29, and at the Board's charter school committee meeting on May 27.

In short, I am recommending that the Board adopt a three-to-one ratio of achievement to growth (75 percent achievement, 25 percent growth). If you accept my recommendation, this will increase the weight given to growth compared to the four-to-one ratio (80 percent achievement, 20 percent growth) that we have employed to meet our statutory obligation to identify the lowest 20 percent of schools. Regardless of the achievement-to-growth ratio that the Board adopts, this will be the first time that growth is a factor in the identification of the lowest ten (10) percent of districts for purposes of the charter cap statute.

Background

The 2010 Achievement Gap Act included an increase in the cap on charter school seats in the lowest performing ten percent of districts. The Board adopted regulations at the time to define the calculation of the lowest performing districts, based on a two-year average of MCAS achievement for English language arts (ELA) and mathematics.

In 2012, we received a waiver from the U.S. Department of Education allowing us to replace the formula (adequate yearly progress, or AYP) required under the federal No Child Left Behind (NCLB) legislation to identify schools in need of targeted assistance. The Board adopted regulations establishing the five-level accountability system that we currently use. Science was added to ELA and mathematics, and a new metric, student growth percentiles ("growth"), was incorporated. This new metric measures the increase in student achievement from year to year. Achievement and growth were weighted at 80 and 20 percent, respectively. The calculation of the lowest performing districts for the purpose of the charter cap lift did not change at that time and, until now, has been based solely on achievement.

In 2013, the Board began consideration of amendments to the charter school regulations. The proposed amendments included a provision that would align the calculation of the lowest performing districts to the calculation used for determining the lowest performing 20 percent of schools, to the maximum extent permitted by law. After significant discussion and numerous opportunities for public comment, the Board approved the amendments in March 2014. This meant that science scores and student growth percentiles would be incorporated into the cap lift calculations for the 2014-15 charter application cycle.

In conjunction with consideration of amendments to the charter school regulations, we have discussed with you proposed changes to the 80/20 achievement-to-growth ratio used in our current school accountability formula. Under our current regulations, any change in the ratio will affect both our classification of schools for accountability purposes and our calculation of the lowest performing ten percent of districts for charter cap lift purposes.

While we are discussing changes to the calculations employed in the Commonwealth's accountability system, it is important to note that the Department is in the process of applying to the U.S. Department of Education for a one-year extension of our NCLB waiver, per the Board's vote in April 2014 endorsing the extension request. The waiver has allowed us to operate under a unified accountability system. The waiver extension application is separate from the selection of the achievement-to-growth ratio. We are hoping for approval of our waiver extension request, without which Massachusetts would have to return to pre-NCLB waiver days when we evaluated schools and districts by two different sets of metrics (AYP for purposes of federal accountability requirements and our Massachusetts-specific five-level system for classifying performance). I will keep you posted on the waiver request.

Continuing Discussion

Our lengthy discussions on this issue reflect both the importance of the policy and the strong views of many of our stakeholders. I know that each of you has received a fair amount of correspondence on this subject since our last meeting, from those who would like us to reduce or eliminate the use of student growth percentiles and from those who would like us to give growth even greater weight.

In reaching my recommendation, I struck a balance among three foundational principles:

  • The formula should reflect fair and ambitious, but achievable targets for schools and districts. The addition of student growth as a factor in our formula acknowledges that not all students enter a school at the same level of performance. I have not been persuaded by the arguments that growth should be eliminated from the formula.

  • The formula should help us direct our limited resources for targeted assistance to schools that most need them. Some have advocated that district rankings be based solely on growth. Looking at the data, it is clear that giving excessive weight to student growth can distort the identification of the neediest schools and districts. Experience is instructive.

    In the late 1990s, the School District of Philadelphia ranked its 260 schools primarily on growth (attendance and student persistence measures received a small proportion of weight), with no weight given to achievement. The Greenfield School in center city Philadelphia - one of the highest achieving and most sought-after schools in the city - had one of the lowest growth rates in the district. I am convinced that the level of resources devoted to improving that particular school would have been better spent on the many Philadelphia district schools with very low achievement. In addition, the identification of the Greenfield School as one of Philadelphia's low-performing schools undermined the credibility and legitimacy of the accountability system in the eyes of many parents, policy makers, and elected officials.

  • The formula should be based on factors that represent a signal (are of practical significance) more so than noise (are of little practical significance). The greater the level of aggregation (state compared to district; district compared to school; school compared to teacher; teacher compared to student), the lower the degree of variation among median student growth percentiles (SGP). District-level median student growth percentiles cluster close to the 50th percentile, more so than school, teacher, and student level SGPs. As a result, the impact of an 80/20 versus 70/30 ratio is of little practical significance at the district level: a district at the 45th percentile and a district at the 55th percentile growth ranking both are securing growth that is quite close to an SGP of 50.

At the Board meeting I will share with you statistical analyses prepared by our education data services unit that illustrate these principles.

Having weighed these principles and having examined analyses of the impact of changes in the weighting of achievement-to-growth, I am recommending a change from the current four-to-one (80/20) ratio to a three-to-one (75/25) ratio. This change reinforces our message that student growth is a valid metric and gives recognition to schools that are promoting strong year-to-year student learning gains. I am not recommending a larger increase in the weighting of growth, because it would start to distort the identification of the schools and districts that are most in need of our attention and assistance. While we applaud strong growth, achievement that ensures student readiness for success at the next level remains our overarching goal.

Final Thoughts

The public debate on the weighting of achievement and growth has tended to focus on the impact on the charter school cap, despite the fact that only a small number of districts would be affected by a change in the cap calculation. In carrying out the Board's role as the state's charter school authorizer, we have always tried to administer the charter school statute fairly and impartially. Here, the Legislature has asked us to create a rubric for comparing district performance on our statewide assessments. I ask the Board to consider the reasonableness of my policy recommendation, without regard to the speculation as to which districts might or might not fall into the lowest ten percent.

Some have suggested that the Legislature intended the 2010 charter cap lift to focus on larger districts with higher percentages of low income and minority students. But the plain language of the statute is that the cap lift is to apply to the lowest performing districts, regardless of size or demographics. I do not accept the proposition that some urban districts must always rank below low-performing suburban or rural districts.

It also has been pointed out, more accurately in this case, that districts that move out of the bottom ten percent as a result of any formula change may have relatively low performance. As we discussed at length at the Board's charter committee meeting on May 27, this metric is not designed to distinguish low performing districts from high performing districts. We are distinguishing among districts that are on the cusp of being in the tenth percentile or below.

If the Board adopts my recommendation, the new rubric will be used to determine the availability of charter school seats during the 2014-15 charter application cycle and to determine the lowest performing 20 percent of schools for the 2014-15 school year. A motion is enclosed for your consideration. I look forward to our discussion at the June 24 meeting.

Enclosure: Motion



Last Updated: June 23, 2014
E-mail this page| Print View| Print Pdf  
Massachusetts Department of Elementary and Secondary Education Search·Public Records Requests · A-Z Site Index · Policies · Site Info · Contact ESE