Measuring Student Progress in Massachusetts – First Statewide Release of Growth Model Reports

In March 2009 I provided an update to the Board on our development of a new way to measure student progress using a percentile growth model. On October 27th, the day of our regular monthly Board meeting, I will be releasing the first round of student growth reports for all districts and schools. This memo provides you with an update on the reports that will be released and the insights we are gaining from this new measure. At our October 27th meeting we will provide you with additional information and an opportunity to discuss the growth model further.

The model enables us to report the extent to which student performance, as measured by MCAS, changes over time. The percentile growth metric is an important complement to the year-by-year test scores, since it takes into account each student's prior achievement when calculating progress. The system relies upon results generated from testing all students in grades 3-8 and high school in reading and mathematics on an annual basis, which has been required by the federal No Child Left Behind law since 2006.

Here is a brief summary of how the growth model works. For most tested grades, the model uses two prior years of MCAS scores to establish a cohort of "academic peers" for each student; this is done separately for English Language Arts (ELA) and Mathematics. We then compare the scores earned by students in the third year to those earned by all other students in their cohort and calculate a "student growth percentile" (SGP). As an example, let's imagine that "Gina" is in a cohort of students who, like her, earned a score of 230 in ELA in fifth grade and 230 in ELA in sixth grade; these are Gina's academic peers. In seventh grade, Gina once again scored 230 in ELA which is higher than the scores of 35% of her academic peers and lower than 65% of her academic peers. Gina has a student growth percentile of 35 for ELA in seventh grade. In other words, most students with similar prior achievement to Gina gained more than she did.

The Department conducted a field test of the growth model with a diverse group of nine districts from April 29th to July 2, 2009. The participating districts were: Community Day Charter Public School, Franklin, Lowell, Malden, Newton, Northampton, Sharon, Springfield, and Winchendon. The field test has helped confirm some of the Department's initial findings and has also revealed important discoveries. For example:

• In one school district, the growth model revealed that what had appeared to be modest gains in MCAS performance from grade 3 to 4 actually represented a very low student growth profile. The district explained that the children changed schools when they transitioned from 3rd grade to 4th grade and school leaders were in discussions about whether they were sufficiently supporting this transition. Impact: The district ultimately reconfigured the grades so children were not changing schools during the challenging transition from "learning to read" to "reading to learn."
• In another district, the growth model revealed that typical to high growth was made at all grades tested except the 5th grade. The district explained that a new curriculum had been introduced that year without any staging of its implementation. Impact: The review of these performance data led to discussions at the district level about the strategies they should use when introducing new curricula.

I want to thank the nine pilot districts for their generous contributions of time, talent and insights during one of the busiest periods in the school year. Districts and schools across the state will benefit from what our staff learned from this process including countless revisions to reports, training and interpretive materials. All districts and schools will be provided with secure online access to:

• Student Growth Rosters
• Student Growth Scatter Plots
• District Growth Scatter Plots
• District Growth Stacked Bar Charts
• District Summary
• Quick Start Guide
• Massachusetts Student Growth Percentiles Interpretive Guide
• Student Growth Percentiles Tutorial Video

Additional resources available to districts and schools include six "Data Warehouse Professional Development Courses," including a focus on data-driven decision-making and continuous improvement. The Department has pre-selected vendors so that districts can more easily procure these trainings for their staff.

The public and policy leaders will be provided with aggregate results that will be integrated into the School and District Profiles on the public website. Beginning October 27th, you will be able to find school and district growth data by clicking on the following link: http://profiles.doe.mass.edu/state_report/mcas.aspx.

The support materials referenced above, including the video, will be available at: http://www.doe.mass.edu/mcas/growth.

I am excited about the opportunity that this new measurement of student growth presents for the improvement of curriculum and instruction in all districts. In addition, the growth measure in combination with MCAS proficiency rates will provide a more robust profile of school and district performance. I am looking forward to our discussion of the student percentile growth model and what we are learning from this new window into student performance.