Report of the Follow-up School Panel Review of the Van Sickle Middle School, Springfield, MA

Introduction

The Rebecca Johnson School was one of twelve middle schools referred for panel review in the spring of 2001 as a result of critically low levels of student performance on State MCAS assessments in 1998 and declining MCAS results in 1999-2000. A panel review was conducted in March of 2001. At that time, the panel found there to be significant inadequacies in the Rebecca Johnson School's plan for improving student achievement at the middle school level. Moreover, panel members found:

At the present time the conditions necessary for successful implementation of a sound plan to improve student performance at the Rebecca Johnson School — effective school leadership and faculty commitment to the school improvement plan— are not in place.¹

Upon consideration of the panel's findings, the district decided to restructure the Rebecca Johnson School, changing it from a K-8 school to a K-5 school, and to move all Rebecca Johnson students in grades 6-8 to the Van Sickle Middle School. The Van Sickle Middle School opened as a brand new facility in September of 2001

Based on the panel findings and changes made by the district, the Commissioner deferred action on the determination of under-performance for a period of six months, and provided a $25,000 grant to support planning and school improvement efforts during that time. A follow-up review was conducted at the Van Sickle Middle School by a four member review panel at the conclusion of the six month deferral period.

The Scope of the Follow-up Review Process

The follow-up review panel's charge was to review the original panel's findings, analyze current data and written information on the school's improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner of its findings relative to the same two key questions that guided the original panel review:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan?

The follow-up panel's responses to the two key questions that defined the scope of their review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to them. A list of panel members who participated in the Van Sickle School follow-up review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration in determining whether the Van Sickle School is deemed to be under-performing.

MCAS Test Results

Cycle I Results (Rebecca Johnson)

Overall, grade 8 MCAS results have declined in all content areas since 1998. Comparing the 1998 baseline score to the 1999 and 2000 average, scores declined one point in English Language Arts, three points in Mathematics, and one point in Science & Technology. In 2000, 41 percent scored in the Failing performance level in English Language Arts, 86 percent scored in Failing in Mathematics, and 83 percent scored in Failing in Science & Technology. MCAS participation rates decreased in 2000, especially in ELA where approximately 11 percent of the regular and special education students were exempted from participating.

2001 Results (Rebecca Johnson)

A= Advanced
P= Proficient
NI= Needs Improvement
W= Warning (formerly Failing)

Panel Reponses to the Key Questions

The questions for evaluating school performance for the panel review process assume that a school is operating with a School Improvement Plan developed to meet the needs of an existing population of students. This report serves as a review of the Van Sickle Middle School (VSMS) in its sixth week of operation. The school has not yet developed a School Improvement Plan, but is operating with an Instructional Plan developed by the principal in consultation with key district staff. Panel responses to the Panel Review Questions acknowledge the Instructional Plan as the school's current plan for improvement while recognizing that this plan does not meet the expectations of a School Improvement Plan for establishing specific objectives, benchmarks, timelines, and evaluation procedures.

Key Question 1: does the School have a Sound Plan for Improving Student Performance?

While the Van Sickle Middle School has yet to develop its school improvement plan, the school's principal, in consultation with key district personnel, has developed an Instructional Plan that sets forth a viable, research-based vision and framework for school improvement. This Instructional Plan reflects a strategic step toward the development of a sound school improvement plan.

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

The school principal and district staff analyzed available data, including external and internal test data, along with staff and programmatic data to identify the gaps in student and school performance that were likely to have led to the poor performance of middle school students at the Rebecca Johnson School. The primary gaps identified included very low performance in reading and language arts, a lack of teachers certified in the necessary content areas, and lack of curricular and programmatic features needed to address the learning needs of middle school students.

Prior to the opening of the VSMS, the Principal and district administrators analyzed available test data to identify specific trends and gaps in student performance. Test data included: 1) available MCAS results - scores from the 4th grade English Language Arts, Math, and Science tests for 6th, 7th, and 8th grade Van Sickle students; 2) Spring 2001 ITBS scores for grades 6-8 in Reading, Language, and Math; and 3) mid-year district assessment data. From this analysis, they identified student literacy skills as a significant weakness to be addressed as a priority for planning the VSMS program.

In addition to analyzing student performance data, the Principal and district staff looked at staff qualifications data and found a clear gap in content area certification among the middle school teachers at the students' prior school. Discussions were also held in order to examine school curriculum, instruction, and organizational systems in light of student outcomes.

From these discussions, there were several areas of need that were determined to be critical in the formulation of the Van Sickle plan. These included hiring middle school staff certified in their content areas, implementing structures that would provide support to middle school students and teachers, creating schedules that would support literacy blocks, creating a new model of instruction for English language learners, and setting forth policies and practices that promote clear expectations for students and staff.

While the particular needs of middle school students attending Van Sickle were considered as described above, the district was also working to identify needs in all of the district's middle schools. This work led to the identification of a lack of academic rigor and high expectations as underlying problems to be addressed district-wide. Also identified as an underlying cause for poor performance was the prevalence of a focus on teaching rather than on learning. Thus, while several efforts were underway to improve the quality of teaching, there was little attention given to what students were actually learning. The involvement of the VSMS principal in these district discussions resulted in the VSMS Instructional Plan addressing priority needs of the district as well as those that are particular to the school.

Given the fact that the school was in its sixth week of operation at the time of the visit, there was much discussion of data that staff plan to consider in monitoring student progress as they move forward. A student writing assessment was given in October, and the results of this writing assessment will be used to establish a baseline for assessing writing progress throughout the year. Teachers across the district are being trained in a protocol for Looking at Student Work and will use this protocol to analyze this writing assessment. Mid-year and end-of-year district assessments will also be used on an ongoing basis to analyze student performance. These assessments are used at the district and school levels to identify curricular gaps and areas of strength.

B. Does the plan set out specific improvement objectives that are grounded in the school's analysis of the reasons for poor student performance?

While the school's Instructional Plan describes key goals and school improvement initiatives designed to improve teaching and learning at VSMS, it does not set forth specific improvement objectives tied to those goals and improvement initiatives. There are several key improvement initiatives that are set forth in the Principal's Instructional Plan. These include a focus on literacy throughout the curriculum, literacy blocks of instruction with leveled reading groups, teaching teams and student houses, a Sheltered English approach for 6th and 7th graders, an inclusion model of TBE for 8th graders, a highly structured discipline policy, professional development and parental involvement, and expectations related to setting a climate of high expectations of students and staff.

While the plan is a well-thought out, carefully crafted framework that focuses on teaching and learning and addresses the supports and expectations needed for successful implementation, it does not in its current form provide specific objectives and plans for evaluation. It is not yet clear what the school's academic standards are nor how student progress toward them will be measured. While teachers were observed using the Massachusetts Curriculum Frameworks and an aligned district scope and sequence to guide their teaching, specific expectations for student performance are not clear and are currently determined by individual teacher teams. The Superintendent discussed district efforts that are underway to specify grade-by-grade expectations.

It is also not clear how and when progress in each of the key improvement areas will be measured. Some of these are very clearly specified, such as the school discipline policy, while others are less defined. In the area of parent involvement, there was less specificity of policy and inconsistent use of strategies to involve parents. For example, some teachers send home weekly notes to parents while others do not.

The school has not yet set school-wide expectations in each of their key improvement areas and determined how they will measure progress in these areas over time. For example, the school researched and selected the Sheltered English approach to language learning for their 6th and 7th grade students. The specific implementation objectives are not clear nor are the plans for evaluating student and program progress. Programmatic and learning goals for the immersion TBE approach for 8th graders are even less clear.

The school Principal and the Superintendent stated a need to move forward with a school improvement planning process in the near future.

C. In order to accomplish each improvement objective, does the plan specify strategies that appear likely to lead to improved student results?

The VSMS Instructional Plan incorporates several strategies that are likely to lead to improved student achievement, particularly in the area of literacy. The plan includes a thoughtful and strategic combination of curricular and instructional initiatives supported by an array of policy, human resource, philosophical, and structural strategies designed to support those initiatives.

Rather than merely setting forth a list of improvement activities, the Principal's Instructional Plan is a coherent, comprehensive, and philosophically consistent framework to guide teaching and learning and to provide support through organizational structures and systems. Teaching and learning initiatives are clearly focused on improving students' literacy skills. All teachers are learning how to incorporate literacy into their instruction through team based collaboration and the use of writing across the curriculum strategies (C.R.I.S.S. program). In addition, the school will be using a district wide protocol for looking at student work. All teachers will receive training in the protocol. Students are being placed into literacy groupings according to their testing data, which indicates their current grade level equivalent. Students with greater needs (below grade level) are given more support through lower than average student-teacher ratios. Teachers will be learning a specific protocol for examining student work as a means to assess performance and improve curriculum and instruction to meet individual needs.

Supporting the focus on teaching and learning is a policy and organizational framework that sets forth expectations for professional development, collegial engagement, parent involvement, and support for students. Strategies include: 1) a schedule that supports an extended literacy block and common planning time for teachers, 2) a house structure that supports the research-based notion of smaller learning communities for students, 3) a literacy focus with a 3-tiered approach based on student literacy levels, 4) a focus on student work and the use of assessment data to drive curriculum and instruction, 5) a sheltered English model for 6th and 7th graders and an immersion TBE model for 8th graders, 6) time set aside in team meetings for professional development and communication with parents, and 6) creating a school climate with clear and consistent expectations for an academic focus.

All of the strategies included in the Instructional Plan are research-based practices that were crafted into a coherent framework for school improvement at VSMS. Supporting the Instructional Plan, the school's discipline policy has already had a positive affect on students and staff. Former Rebecca Johnson students acknowledged that "it is much stricter here…and it is a good thing; it is much clearer and calmer." Supporting the success of this policy is a "do whatever it takes to help students" approach. As an Assistant Principal said, "What we communicate to students is that you are too important for me not to care. " Students and parents said that teachers stay late to help students who need it and that administrators will take time to help individual students, as well as teachers.

D. Are the school's written improvement planning document (s) clear and specific enough to guide their implementation of planned improvement initiatives?

There is not ample specificity in the Instructional Plan to drive implementation and evaluation of the school's initiatives in a clear and coherent fashion. The development of a school improvement plan with specific actions, timelines, and evaluation plans tied to the school's Instructional Plan will be needed as the school moves forward. Please see above "Important Note" for information about this question.

The superintendent's district plan, A Culture of Achievement, sets forth benchmarks for student performance in the MCAS. This plan goes further to state that the district will make greater than average gains in the performance of Black and Hispanic students in an effort to close the achievement gap of minority students. There are no benchmarks in the VSMS plan that can be linked to those district benchmarks.

E. Was the School Improvement Plan developed through a process that will support its successful implementation?

In the context of the opening of a new school, efforts to both develop the school's Instructional Plan and to ensure support for the plan were highly effective. The Principal engaged district staff in its development, ensuring the plan was aligned to district goals and would in turn be supported by the district. The Principal reached out to students and parents at the Rebecca Johnson Middle School to talk about the new school. In interviews with all VSMS teachers and staff, the Principal ensured that all prospective staff would understand, support, and commit to the plan.

Other elements that have led to the high level of support that the school has for the plan include the Principal's clear articulation of the plan, her professional engagement of staff in working to support the plan, and in a steadfast and observable commitment to the well-being of students and staff.

Key Question 2: Are the Conditions in Place for the Successful Implementation of the Improvement Plan(s)?

The leadership provided to date by the Principal of the VSMS has positioned the school well for successful implementation of improvement efforts. This leadership is characterized by reflective and strategic decision-making and an ability to engage effectively with staff, students, parents, and district administrators. All stakeholder groups voiced strong support for the school's leadership and a shared commitment to improving the school.

A. Does the school have effective leadership and sound management?

All sources of evidence considered in this panel review indicated that the Principal of VSMS is providing highly effective leadership and sound management. In a very short time, she has crafted a thoughtful and strategic plan to unite and guide the school, hired a competent and committed workforce, opened a new school building, and created a climate of high expectations and collegiality throughout the school. Her style is characterized as reflective, astute, strategic, and involved. Staff members across the school describe the entire administrative team as an effective, supportive, and involved team.

The Principal has artfully implemented leadership strategies that have created a very positive and unified force for school improvement and student success at the school. These include:

• Reaching out to students and parents of the closed middle school program;
• Hiring certified teachers that are committed to the Instructional Plan;
• Engaging with staff in a spirit of teamwork and professional inquiry;
• Working in the trenches with students and staff;
• Knowing when to push and when to step back;
• Being visible and available;
• Making every decision count.

The panel found several instances of small and large actions and decisions made by the Principal that were all tied to a central purpose and vision. Whether it was making popcorn for the staff or using a metaphor in a presentation (the V geese formation for working together at Van Sickle), the Principal demonstrated a facile ability to make all decisions, whether large or small, important ones. As one staff member said, "She does all the right things to make this a great school."

All students, staff, parents, teachers, and administrators interviewed voiced confidence and praise for the leadership provided at VSMS to date. Their comments were numerous but reflected in the following few. "The leadership is tops." "They [administrators] are in it with us." "The Principal has a clear and humane way of communicating." "People understand her vision and goals — they bought in." "The expectations are clear, and we know our role in it."

B. Is there evidence that the school's faculty supports the planned improvement efforts?

All school faculty and staff, as well as several key district administrators expressed a high level of support for, and confidence in, the school's Instructional Plan and in the progress made to date in moving the school forward. Teachers believe in the goals of the plan and in the leadership that is in place to help move it forward. Staff pointed to positive changes that were already observable in the efforts that are underway. Students articulated improvements in the behavior of their classmates overall as well as a shift in student attitudes toward learning. These early successes are providing feedback to staff that they are on the right track.

Teachers engaged with the panel review process in a fashion that connoted a commitment to continuous school improvement. Teachers were willing to share areas of progress as well as areas needing improvement.

C. Is the school receiving adequate guidance and support from the district leadership?

The VSMS has received a substantial level of involvement and support from district leadership in the start-up of the new school, in the analysis of student needs, and formulation of a plan to address those needs. There is close alignment between the philosophy and goals articulated by the district and those articulated by the leadership of the VSMS. All stakeholders from the district down to the students are articulating an understanding of a common vision and sense of purpose for the school. The district is also a key support in the professional development provided to, and planned for, the VSMS faculty and staff.

District staff collaborated with the Principal in identifying needs and developing plans to address them prior to the school's opening. The vision and goals set forth by the Superintendent helped to serve as a guide for the VSMS planning, especially with regard to creating an academically rigorous middle school with high expectations for student learning. The district has been supporting leadership for school improvement through a three-year partnership with the University of Pittsburgh. The Principal of VSMS has participated in this professional development in her prior and current administrative position.

Staff voiced concerns that they will continue to need district support as they move forward. They do not want to see the support wane when the urgency of the school start-up is no longer serving as a catalyst for district attention.

Conclusion

In its sixth week of operation, the Van Sickle Middle School has both a sound Instructional Plan for improving student performance and a high level of capacity for successful implementation. The school's Instructional Plan is a coherent, comprehensive, and philosophically consistent framework that has provided both a research-based set of strategies for school improvement and a clear direction and sense of purpose for staff. As the school moves forward, clear objectives with benchmarks and methods for ongoing evaluation will be needed to ensure the school has the means for measuring the progress of students, as well as school progress toward the broad goals set forth in the Instructional Plan.

Appendix A
Team Members

Carol Keirstead, Panel Chairperson, Senior Research Associate, RMC Research Corporation, Portsmouth, NH

Janice Johnson, Quadrant Manager, Worcester, MA

Kathy O'Connor-Abrams, Assistant Principal for Academics, Chelsea High School, Chelsea, MA

Tom Buffet, Consultant for SchoolWorks, Beverly, MA

Appendix B
Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

¹ Quoted from the Commissioner's letter to the Rebecca Johnson School dated April 26, 2001.