The Massachusetts Board of Elementary and Secondary Education

Level 4 Schools: Third Year Insights and Decisions

---

At the special meeting of the Board of Elementary and Secondary Education (Board) on November 26, 2012, Department staff presented the framework being developed to inform fall 2013 decisions for schools designated Level 4 ("underperforming") in 2010. For each of the 34 Level 4 schools designated in 2010 (and consistent with statutory responsibility and authority), I will be making determinations this fall about whether the school will be designated Level 1-5 based on student performance trends and evidence of school and district systems to sustain progress. The November 26th presentation focused on the criteria being used to make these determinations as well as an overview of how the Level 4 schools were progressing based on two years of data.

For the Monday, June 24 special meeting of the Board, we will provide an update on the decision process and take a closer look at Level 4 schools at either end of the performance spectrum: those that have seen significant gains and may exit to a Level 1, 2 or 3 status and those that have continued to struggle and are most "at risk" of being identified as Level 5. More specifically, we will:

• Present recent research about effective school and district practices in the first cohort of 34 Level 4 schools and hear from the school and district leadership of one of these rapidly achieving schools; and
• Discuss the decision-making and intervention options for Level 4 schools most at risk of being identified as Level 5. We will hear from two districts that have entered into partnerships with state-approved "turnaround operators" in order to intervene in their at risk schools in the 2013-14 school year.

This memorandum will provide background and summary information on these topics.

I. Background: Level 4 Schools

The 35 Level 4 schools announced in March 2010 were the first schools to undergo a new turnaround plan process defined in An Act Relative to the Achievement Gap, the landmark legislation signed into law by Governor Patrick in January 2010. This statute provided new flexibilities to turn around our state's lowest performing schools. Each of the nine districts with one or more of these Level 4 schools (Boston, Springfield, New Bedford, Fall River, Holyoke, Lawrence, Worcester, Lowell and Lynn) was required to produce a turnaround plan for the Commissioner's approval and was given priority to apply for competitive federal School Redesign Grants of approximately $500,000 per year for up to three years for each of its Level 4 schools. The Department also provided targeted supports to the schools and helped connect them to additional turnaround resources where appropriate. (One of the identified Level 4 schools in Boston subsequently was closed, leaving 34 schools in first cohort.)

The Department has assessed progress and conducted research on the Level 4 schools to help identify and share effective practices, as well as to identify whether schools were on track to meet their identified goals. This research has informed district and school strategies to enhance rapid improvement in the turnaround schools. The continuing tracking of performance has been used to gauge progress and focus state and local interventions during the course of the turnaround plan terms.

At the expiration of each school's three-year turnaround plan, the Department is required by state law to "conduct a review of the school to determine whether the school has improved sufficiently, requires further improvement or has failed to improve" (M.G.L. Ch. 69, Section 1J (l)). The 2012-2013 school year is the third year of turnaround plan implementation for these 34 schools and, once 2013 MCAS performance results are available in early fall, the Commissioner will determine next steps for each school.

The Department is collecting evidence through a combination of data analysis and school and district reviews to determine each Level 4 school's progress. Once this evidence is collected and analyzed for each Level 4 school, I will determine that:

• the school has improved sufficiently for its designation as Level 4 to be removed; or
• the school has improved, but remains underperforming and should stay in Level 4; or
• the school has not improved sufficiently and should be designated as a Level 5 ("chronically underperforming") school and placed under receivership.

(See Attachment 1 for relevant excerpts from the law and regulations about this process.)

In order to create a shared understanding with key stakeholders about the Department's decision-making for Level 4 schools and planning for potential identification of Level 5 schools, we have discussed the process and planning with the following groups:

• Representatives from state leadership at the Massachusetts Teachers Association (MTA) and American Federation of Teachers - Massachusetts (AFT-MA) (conference call on September 19, 2012);
• State leadership from the MTA and local teachers union presidents from several affected districts (meeting on December 19, 2012); and
• The Board's Accountability and Assistance Advisory Council (meeting on February 13, 2013).

II. Research on Effective School and District Practices

In April 2012, we reported to the Board that we were encouraged that 22 of the 34 Level 4 schools achieved significant gains in both English language arts (ELA) and mathematics after the first year of turnaround efforts. A year later, after two years of turnaround, we remain encouraged that student achievement data results show that at least 16 of the 34 schools are on target to meet their ambitious improvement goals.

In 2011, the Department commissioned an on-going study by the Institute for Strategic Leadership and Learning (INSTLL, LLC) designed to identify the practices in Level 4 schools that appear to be contributing to rapid improvement in some schools. Two reports have been produced:

1. In April 2012, the Board received a report on the first-year findings entitled, "Emerging Practices in Rapid Achievement Gain Level 4 Schools." The key findings in the first report identified three significant commonalities among Level 4 schools with rapid achievement gains:

   1. The school has established a community of practice through leadership, shared responsibility, and professional collaboration.
   2. The school employs intentional practices for improving teacher-specific and student-responsive instruction.
   3. The school is able to provide student-specific supports and interventions informed by data and the identification of student-specific needs.

   The report also confirmed that each of these three emerging practices was grounded in a safe, orderly, and respectful environment for students and teachers.

2. The year two report of this continuing study of the first cohort of 34 Level 4 schools, "Emerging and Sustaining Practices for School Turnaround - 2013", is presented to the Board today. (See Attachment 2 for the report.)

   In this second report, the evaluators identified 13 Level 4 schools that made the most rapid gains and compared them with 9 schools that had experienced minimal gains over two years. These 22 schools were analyzed to explore trends and themes that may explain the differences in achievement gains. Through a detailed analysis of school practice and implementation, the report confirms that the findings identified in the first report continue to emerge as key indicators of success.

   In the most recent analysis, a new finding of importance notes: "There appears to be a strong relationship between schools' initial, single year teacher turnover rates and ability of schools to make rapid improvement." Data suggests that higher rates of teacher turnover appear to be linked with schools' success in meeting their improvement goals. It is important to consider, however, that while there appears to be a strong relationship between teacher turnover rates and a school's early achievement trajectory, the meaning of this relationship is not always clear. For instance, teachers may voluntarily leave a school because improvement efforts are lagging or the school leader may be encouraging particular teachers to look for work elsewhere. Additionally, district policies (e.g., bidding and bumping, other hiring practices) may be influencing turnover rates. Further document analysis and additional interview data are necessary to better understand how teacher turnover is related to school turnaround efforts. Our evaluator will look more closely at this finding as we continue to learn from our Level 4 schools' experiences.

3. Finally, additional research is attached, entitled "School Redesign Grant - Management Briefing Memo - 2013." This report is focused on districts' systems of support for their low performing schools. This briefing memo presents preliminary findings from an ongoing evaluation being conducted by the UMass Donahue Institute of the Department's School Redesign Grant (SRG) program. Here, we are learning how the emerging best practices identified in our first cohort of Level 4 schools are prompting districts to shift/change/create systems that build on these best practices in order to impact improvement in all their high need schools. (See Attachment 3 for the memo.)

   While the final report is not expected until late summer, preliminary findings are quite encouraging. The 2013 evaluation has concluded that, in contrast to previous evaluation findings (2012 report) which suggested that districts' capacity to provide coordinated, systematized, effective support to redesign schools was under-developed, the more recent round of findings reveals notable improvements. For instance:

   • Support for school leaders: Some districts have created specific mechanisms to support the principal as an instructional leader, including the establishment of school-based teams, targeted use of data, and embedded professional development to build principals' skills and knowledge.
   • Consideration of school context: Some districts are providing increasingly differentiated supports to schools, which suggests they are considering the context when thinking about implementing reforms.
   • Data management: Some districts have modified their approaches to data management and use in order to support schools more effectively. There is growing evidence that districts are becoming more adept at using data to assess schools' needs, to foster and facilitate necessary (if difficult) conversations, to challenge assumptions and/or to correct erroneous inferences, and to encourage effective, data-driven budgeting. In some districts, efforts are made to strengthen principals' and others' skills with respect to data through embedded professional development opportunities.
   • Strategic use of partners: Some districts are demonstrating an increasingly proactive, thoughtful, and strategic approach to deciding whether and/or how to use external partners' services to support their improvement goals.
   • Planning for scaling up: Some districts are demonstrating an increased ability to scale-up: replicating success, but with an eye toward context.
   • Reorganization at the district level: Some districts are pursuing reorganizations, including the creation of new positions. Some are focusing now on recruiting and hiring for turnaround schools. These developments highlight changes in the strategies and structures that districts deploy to support schools.

Boston Superintendent Carol Johnson and Principal Andrew Bott, Orchard Gardens K-8 School, will join us at the special meeting on June 24 for a brief discussion of their work with this rapidly improving Level 4 school.

III. Decision-Making and Interventions in Level 4 Schools Most "At Risk" of Being Identified as Level 5

Options for Identifying and Intervening in "At Risk" Level 4 Schools

As described in Section I (page 2) of this memorandum, the Department is collecting evidence through a combination of data analysis and school and district reviews to determine each Level 4 school's progress. In cases where the school has achieved little or no progress, the Commissioner may make a Level 5 ("chronically underperforming") school designation and place the school under receivership.

The Act Relative to the Achievement Gap Act affords multiple pathways through which the Department can intervene in a Level 5 school. It allows "the Commissioner to send a targeted assistance team to the school to assist the superintendent with the implementation of the turnaround plan, require the superintendent to implement the turnaround plan, or select an external receiver to operate the school and implement the turnaround plan," and that an external receiver, if appointed, "shall have full managerial and operational control over the school."

If a Level 5 school designation is needed, the Department will carefully examine the district and school conditions that have contributed to the lack of improvement and determine which of the above options is most likely to be successful in that specific context. The Commissioner will then make recommendations to the Board about a Level 5 school designation.

Recruiting, Screening, and Selecting "Turnaround Operators"

In order for the Department and school districts to have a full range of options to intervene in Level 4 and 5 schools, we have spent the last two years recruiting and vetting "turnaround operators"- organizations that can assume management of underperforming schools and turn around performance quickly and consistently. Through our Race to the Top "Turnaround Operators Initiative," the Department released a Request for Responses to recruit, screen, and identify potential school turnaround operators with a track record of success. To date, seven operators have been approved and added to our portfolio of providers. (See Attachment 4 for a list and brief description of the approved turnaround operators.)

The Department has worked closely with the approved school turnaround operators to build their capacity to engage with underperforming and chronically underperforming schools throughout the Commonwealth. Examples of this support include: assistance in identifying the conditions necessary to successfully implement their models, facilitating introductions with potential districts and schools, and providing some funding to build capacity.

Communication with District Leaders about "At Risk" Level 4 Schools

In the fall of 2012, the Department scheduled meetings with the superintendents of five districts (Boston, Fall River, Holyoke, New Bedford, and Springfield) to discuss the struggling Level 4 schools in their districts - Level 4 schools that were showing little or no improvement after two years of turnaround. The meetings were framed as "strategy sessions" for accelerating improvement in these schools, and it was suggested that superintendents invite school committee members and union presidents to participate.

After sharing Department-developed analyses of each Level 4 school's progress toward meeting their measurable annual goals, the Department explained the strategies we would likely consider for Level 5 school receivership - including the appointment of proven school turnaround operators to manage the schools. The purpose of these meetings was to deliver a clear message that, unless significant progress was made in these Level 4 schools during the 2012-13 school year, the districts run the risk that the schools will be designated Level 5 in the fall.

We emphasized that, if district and school leaders were confident that the current turnaround efforts in these schools would yield significant gains during the 2012-13 school year, they could choose not to take alternative action this year and "stay the course" of turnaround plan implementation. However, if leaders were not confident that the current turnaround efforts would produce significant gains in student performance, we offered that they could elect to engage with one of the state-approved turnaround operators, thus initiating an option the Commissioner would consider at the time of a potential Level 5 designation. This voluntary option would have the dual benefits of:

• establishing a new plan for the students that would begin in the Fall 2013, and
• eliminating the need to designate the school as Level 5, provided that the intervention is as rigorous as what the Commissioner would use at Level 5.

Districts were asked to consider the available information and follow up with Department staff for further assistance.

District leaders from Boston, Holyoke, and Springfield have made the decision to enter into partnerships with state-approved turnaround operators: Boston with Blueprint Schools Network; Springfield with the Education Innovation Laboratory at Harvard University (EdLabs); and Holyoke with Project GRAD USA. Conversations are currently underway in each of the districts to finalize details of the partnerships. The Commissioner, along with Department staff, is reviewing each agreement to ensure that the conditions for the turnaround operator are consistent with the rigor and flexibilities the operator has leveraged in previous successful partnerships. Once the agreements are finalized and the partnerships are being implemented, both the district and operator will be required to participate jointly in quarterly progress meetings with the Department to ensure that both parties are delivering on their commitments and educational conditions are rapidly improving in the schools. Fall River Public Schools has made the decision to close their "at risk" Level 4 school, the Henry Lord Middle School, and New Bedford Public schools chose to continue with their Level 4 school turnaround plan in their "at risk" Level 4 school, the John Avery Parker elementary school.

Representatives from EdLabs and Project GRAD will be at the special meeting on June 24th along with the superintendents from Holyoke and Springfield Public Schools to talk more about their models and provide updates on the status of district partnerships.

Enclosures: