District/School Administration Administration
The Massachusetts Board of Elementary and Secondary Education

Measuring Student Progress in Massachusetts - Development of the Growth Model

I am writing to update you on the Department's development of a growth model. The model makes use of an important new metric we are planning to use beginning in the fall of 2009 to report the extent to which student performance, as measured by MCAS, has changed over time. This metric is an important complement to the year-by-year test scores, since it takes into account each student's prior achievement when calculating progress. At our March 24th meeting, Deputy Commissioner Jeff Nellhaus will explain the derivation of the metric; illustrate how it can be used in reports for individual students, schools, and districts; and describe how we intend to pilot the metric this spring in several districts.

Since 2003, when we began testing all students in grades 3-8 and high school in reading and mathematics on an annual basis, we have intended to develop a metric to report change in student performance over time to complement existing MCAS reports of student performance. Analyzing and reporting student growth can enable better decision-making on many levels. Such a metric can, for example, be used to:

• improve curriculum and instruction;
• make sound instructional decisions for individual students;
• inform school and district accountability decisions;
• guide the Department's assistance efforts to districts and schools; and
• conduct program evaluations.

We evaluated several different ways of calculating student progress (see appendix), but all fell short of our needs until this past fall, when we became aware of an innovative, new approach developed by Dr. Damian Betebenner at the National Center for Assessment in Dover, NH. His method is to measure students' progress relative to other students with a similar test score history. This new metric, called a student growth percentile, has already been implemented in the state of Colorado and with the Arizona Charter Schools Association.

With the student growth percentile metric, we can answer the questions such as:

1. What is the typical change in test scores that occurs between any two grades for students with similar prior test scores?

2. For a given student, how does his/her growth compare to students with similar prior test scores? Is it similar to, greater than, or less than typical growth?

3. Which schools demonstrate better than (or less than) typical growth for their students compared to schools with similar overall achievement (e.g., two schools might have similar percentages of students performing at the proficient/advanced levels, but one school shows higher growth from the previous year than the other)?

This information on individual students is useful for teachers and parents who want to know whether their students are progressing. When aggregated to the subgroup, school, or district level, these data can also indicate the progress made with groups of students and the impact of various programs and interventions. For example, we will be able to highlight particularly effective interventions by identifying low performing schools that consistently demonstrate high growth and then share these effective practices with other schools and districts.

Moreover, we can also use a student's growth history to generate projections of how likely that student is to reach proficiency (or any other performance benchmark) within one year, two years, or more. This could be helpful to schools in working with individual students and to our ability to focus our accountability and assistance work on the districts with high numbers of students who not only are not yet proficient but who, in addition, are not on a trajectory for proficiency.

This initiative merits further discussion. Most important, we need to determine how best to ground percentiles-a relative measure-within our standards-based system, so that we maintain high expectations for both the performance and growth for all students. This will require establishing a measure of adequate growth to complement the measures of relative growth we have already developed. This might be growth sufficient to reach proficiency or, for students already at proficiency, growth at a pace to indicate that the student will continue to perform proficiently or higher. We also need to consider whether and how we might propose to incorporate growth into our district accountability system or in determinations of Adequate Yearly Progress. We will seek the Board's input as we move forward in these areas.

In the meantime, the Department is planning to roll out student growth data on a pilot basis with a small group of districts this spring. This will prepare us to release student-level growth calculations for all students, schools, and districts using data from the spring 2009 MCAS testing cycle. The districts participating in the pilot will help with developing reports and interpretive materials as well as with understanding the variety of ways in which the field may find the data useful. We anticipate reporting a range of reports and supports, including:

• student rosters for districts and schools showing growth for each student previously or currently educated;
• aggregated school and district measures of the median growth of their students and subgroups;
• visual displays arraying growth against performance;
• analyses that highlight which schools and districts are achieving particularly high growth, especially among those with relatively low performance levels;
• reports for parents on their children's growth;
• training materials and sessions to help people understand how to use the growth data; and
• training for using the Data Warehouse to access growth data.

I will provide updates to the Board over the coming months with proposals for how to handle the policy questions that arise as we learn more about what the model tells us about student performance in Massachusetts. We look forward to discussing this proposed model and its implications with you.

   Measuring Growth in Student Performance on MCAS