Report of the School Panel Review of the Williams North Middle School, Chelsea, MA

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low and no trend toward improved student performance is evident from MCAS data. The Williams North Middle School was one of twelve Massachusetts middle schools meeting this criterion that were selected for panel reviews in spring, 2001. The Panel Review of the Williams North Middle School was conducted on March 13, 2001.

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan?

The panel's responses to the two key questions that defined the scope of their review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to them. A list of panel members who participated in the Williams North Middle School review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Williams North Middle School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist, or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Williams North Middle School Profile

Williams North is one of two middle schools in Chelsea, with 483 students enrolled in grades six through eight in 2000. In that year, 89 percent of the students were eligible for free or reduced-price lunch, and 73 percent spoke a first language other than English. The school identified less than one percent of its students as Limited English Proficient and reported that 71 percent were Hispanic, 13 percent were white, eight percent were African-American, and eight percent were Asian.

In 1999 (the most recent year's data available to the panel), students missed on average 15 days of school, for a daily attendance rate of 91.8 percent. For the 1999-2000 school year, the daily attendance rate improved to 92.4%. In that same year, 65 students (13.6%) received one or more out-of-school suspensions, while 49 (10.2%) were suspended in school. One student was excluded more than 10 days. In addition, eight students (1.7%) were reported as dropping out, and 25 (5.3%) were retained in grade.

Staffing

Williams North's middle school students are taught by 28 full-time teachers and four "teacher leader/ curriculum facilitators," for an approximate average pupil-to-teacher ratio of 15 to 1. The central staff consists of Principal Anthony M. DiGregorio, an assistant principal, two guidance counselors, and a librarian. Of the 28 middle school teachers, 16 (57%) have attained a master's degree. Of the seven teachers and teacher leaders listed as math or general science instructors, none were reported as holding a bachelor's or a master's degree in their subject (data were not reported for all teachers, however). Three possess math or science teaching certificates.

Mcas Results

In comparing the 1998 MCAS results to the 1999 and 2000 average, scores were level in English Language Arts, while declining four points in Mathematics and three points in Science & Technology. In 2000, 26 percent scored in the Failing performance level in English Language Arts, 74 percent scored in Failing in Mathematics, and 80 percent scored in Failing in Science & Technology. Of the 22 special education students tested in 2000, all scored in the Failing performance level in Mathematics and Science & Technology for a combined average scaled score of 201 in each subject. Another five special education students were exempted from the test.

Panel Reponses To The Key Questions

Key Question 1: does The School Have A Sound Plan For Improving Student Performance?

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

There is evidence that the school has analyzed a variety of data to identify gaps in student performance. They have examined some demographic data, including the student mobility rate between fourth and eighth grades and the average rate of attendance, and identified these as areas of concern. Lead teachers have conducted analyses of MCAS 2000 data in the areas of English Language Arts, Math, and Science and Technology and determined areas of weakness. While the English Language Arts analysis identifies possible reasons for poor performance and leads to a plan of action, analyses of Math, Science and Technology, and English Language Arts Long Composition do not clearly identify weaknesses nor suggest a course of action. Panel members found the math analysis difficult to understand and questioned its usefulness to teachers. The principal reported that they have not analyzed Social Studies results.

The school has also used other information to identify weak areas. Students, parents and faculty completed needs assessment surveys, and from subsequent discussions with faculty, five courses of action resulted that appear to act as a set of improvement steps separate from those identified in the School Improvement Plan. In addition, the district conducts a professional development survey twice a year to determine areas of need and create a professional development plan outlining offerings available during the school year as well as over the summer that respond both to staff's expressed needs and to district priorities. School personnel mentioned the use of other assessments, including intake tests, trial MCAS tests, exit tests, and Iowa test results. However, the panel was unable to determine how these analyses informed their improvement planning process.

The School Improvement Plan states that both school and external factors have affected poor student performance on MCAS. The leadership report elaborates on those factors contributing to existing gaps. Reasons given include the transient nature of the school's population; the alignment of the middle school curriculum to the Massachusetts Frameworks which is a work in progress; the need for more differentiated instruction; the high turnover of teachers and paraprofessionals; and lack of parental involvement. However, no clear link between these factors and the resulting school improvement initiatives is evident.

The principal noted efforts being made to align the existing curriculum with the Massachusetts Frameworks. Last year school personnel focused on articulating minimum standards at each grade level and creating exit tests in each core subject area at each grade level. This year they are working on alignment. The two middle school principals each chair two core subject curriculum committees for the 6-8 grade span. The Superintendent intends to change this model because it presumes both content area expertise in two disciplines as well as curriculum development skills. The panel heard no evidence that the misalignment of particular curricular areas were possible explanations for gaps in student performance that were in turn driving improvement initiatives.

B. Does the plan set out a course of action that is: (1) responsive to the school leader's analysis of the nature and reasons for poor student performance, and (2) is likely to lead to improved student results?

The School Improvement Plan articulates numerous courses of action that are broad and that do not link back to identified weaknesses. Some action steps are outlined in the remediation plan intended to improve MCAS scores. The school concluded upon analysis of MCAS scores that students were unable to read and understand questions asked of them on the test. For this reason, they state that the plan emphasizes improving reading and writing skills. However, the action steps articulated in the remediation plan do not clearly address specific needs identified by the analysis. For instance, one strategy mentioned is to continue implementation of 'Project Excellence', which provides academic enrichment through a before and after school and summer program. However, the focus is unclear since the plan describes it as a comprehensive program that offers classes in math, literacy, science, social studies, and the arts.

The school also outlined a list of 'first steps' identified during discussions of the faculty survey. Some action steps agreed upon include developing a schedule for cluster meetings, forming a committee to examine discipline issues, and expanding the role of students by creating a student council and appointing a student representative to the school improvement team. However, while these steps are in response to teachers' expressed needs, it is unclear how these steps are likely to lead to improved student performance since they do not deal with teaching and learning.

An annotated list of school wide reform strategies from 1999-2000 comprises a third set of action steps within the School Improvement Plan. The review panel did not learn why these were included in the 2000-2001 plan. Some of the strategies relate loosely to the seven identified goals for 2000-2001, i.e. the "Home Room Buddies" program to improve attendance, the integration of special education and bilingual students. As written, however, none of the strategies outline specific action steps that will be undertaken. The plan lacks specific measurable outcomes, realistic timelines for implementation and completion, and checkpoints to monitor progress. Due to this lack of specificity, the plan may be insufficient to yield positive results.

Williams North personnel developed an MCAS Improvement Plan in December 2000 at the request of the Superintendent. It outlines 16 additional "strategies and application" to be implemented during the 15 weeks prior to the next administration of MCAS. Action steps are specific but do not in all cases indicate who is responsible for carrying them out, and timelines indicate only that actions be carried out in the fifteen weeks prior to the next MCAS administration.

There appears to be a good deal of effort directed at providing a variety of remedial programs (e.g. Project Excellence, the tutorial program in reading and math established upon receipt of the Department of Elementary and Secondary Education's academic support grant). However, the school's analysis of data and programs is incomplete, and as a result action steps lack specificity and a clear relationship to weaknesses. Because school personnel have identified general reasons for poor student performance (i.e. lack of reading and writing skills), the plan does not adequately target instruction in specific reading and writing strategies that are likely to improve student performance. For this reason, the review panel determined that the plan may not yield improved student results.

C. Was the School Improvement Plan developed through a process that will support its successful implementation?

The Principal of the Williams North sought the input of staff, parents, and students and used some data and program information to develop the School Improvement Plan. After writing the plan, he brought it back to the School Council for feedback. However, there is mixed evidence of the degree to which the plan reflects priorities expressed by various stakeholders, and the degree to which it was communicated to stakeholders, especially parents. In addition, there are varying levels of commitment to the plan.

The School Council conducted surveys of parents, students and teachers in spring 2000.

One hundred ten parent surveys were returned (44% of the 250 sent out), which reflected 23% of the school's enrollment. Parents were randomly selected, and surveys were translated into the six languages of the school population. The survey results section of the School Improvement Plan revealed that 13% of parents wanted their child to receive additional ESL help though only 10% of the total enrollment is in bilingual classrooms, suggesting that students whose first language is not English continue to need ESL support in mainstreamed classrooms. Parent focus group members also stated that language is a barrier to communication between home and school despite the fact that the school translates notices into families' native languages. They expressed a lack of understanding of MCAS information and anxiety about their child's ability to pass the MCAS test. The panel did not see evidence of language related issues being addressed in the School Improvement Plan.

Two hundred randomly selected students responded to the student survey. As reported in the School Improvement Plan, almost 90% indicated they were generally happy with the school, felt safe at school, got along well with their teachers and classmates, and made progress as a learner. Since the review panel did not see the survey, it is unclear the degree to which it helped inform the school's improvement planning process.

School leadership received a 90% response rate from the teacher survey. The faculty discussed findings at a staff meeting and made additional suggestions, most of which centered around three themes: communication among staff, a systematic method to maintain a safe and orderly environment, and respect among staff and students. A list of action steps (discussed in 1A) resulted from that discussion. Therefore, faculty input appears to have been heard although these steps are a small part of a larger plan. In addition, the Title I office conducted a professional development survey in March to plan summer courses and workshops in response to collective needs. The emphasis, according to the School Improvement Plan, was on content areas and pedagogical techniques.

Parents revealed that some had never seen the School Improvement Plan and did not know if or how it is available. Some teachers reported that while they have the plan, they do not make use of it. Others communicated uncertainty about its contents. Since the plan is the result of incomplete analysis, lacks focus, specificity and all stakeholders' support, the review panel questions its usefulness as a guide for school improvement.

D. Is the School Improvement Plan document clear and specific?

The school has developed a plan that addresses broad areas of concern and lists programs, initiatives and strategies currently in operation as well as some to be introduced during the year. In its present form, however, the Williams North Schoolwide Program Plan is confusing in its layout and lacks essential components. Missing elements include clear statements of problems, thorough analyses of causal factors, desired outcomes, detailed action plans, methods of evaluation, and meaningful timelines.

The School Improvement Plan identifies seven goals for the year. However, these goals exist in isolation from proposed action steps which, as stated earlier (see 1B), appear in at least four places in the plan. The lack of connection between goals and proposed strategies contributes to poor clarity. Furthermore, the relationship between identified problems and planned solutions is not always clear (see 1B). Although there are numerous action steps, they lack the specificity necessary to reach the desired outcomes. It is, for instance, unclear in places who is responsible for carrying out some initiatives and what these initiatives are trying to accomplish. In addition, the school has not identified who will monitor the implementation of initiatives along the way or evaluate the overall effectiveness of its strategies and programs.

The Superintendent of Schools indicated that her leadership team is developing a School Improvement Plan template for next year. In her words, goals need a monitoring piece so that the plan is a roadmap for school improvement.

Key Question 2: Are The Conditions In Place For The Successful Implementation Of The Improvement Plan(s)?

A. Does the school have effective leadership and sound management?

The principal has a positive presence in the building. He has established a nurturing and safe school climate. Panel members observed orderly transitions where faculty led students to and from lunch and monitored hallways between classes. Students communicated that they feel safe in school because of principal, police and security guard presence. An adult stated, "This school is like family." Parents said that school personnel are open and caring. A notable initiative begun at the high school and adopted this year at Williams North is a teacher-student mentoring program. Twenty adults (teachers and administrators) have each taken two to three at-risk students under their wing to give them a personal connection to the school and provide guidance and support.

The principal sought some input from stakeholders in the decision making process. Teachers described their biweekly faculty meetings as open forums where they are free to bring ideas to the table. Comments during teacher interviews and focus groups conveyed that teachers feel supported. The principal also sought feedback from parents and students via the needs assessment surveys, and parents on the School Council expressed their involvement in the school.

Panel members heard no evidence that students take on leadership roles within the building. Teacher survey responses mentioned a desire that the school's leaders find ways to empower students and teachers and inspire ownership and pride in the school. Respondents worried that there is not a strong message of confidence in students' ability to succeed.

Panel members expressed concern that instructional leadership lacks focus and direction to guide improvement efforts that will result in measurable improved student performance. For example, when asked how they monitor progress on School Improvement Plan initiatives, school leaders did not articulate a process to monitor the implementation of action steps. While the Superintendent stated that a template for lesson planning is in the process of being developed, the principal indicated that there is no process in place for reviewing lesson plans. The Superintendent's proposed reorganization plan is an attempt to strengthen instructional leadership by shifting roles and responsibilities so that there are more administrators focused on instruction and supervision.

B. Is there evidence that the school's faculty supports the planned improvement efforts?

There is a willingness on the part of faculty to support planned improvement efforts. Staff input received from the needs assessment and subsequent discussions resulted in five action steps: the development of a cluster meeting schedule agreeable to all; the creation of a staff advisory board; the formation of a committee to examine discipline; finding ways to promote the positive; and expanding the role of students. However, it is unclear how these "first steps" relate to the overall School Improvement Plan, and the panel did not hear the status of these initiatives.

Teacher survey responses also indicated the faculty's support of planned improvement efforts. Numerous teachers agreed that in order to improve student performance teachers need to focus instruction on math and English language arts, make use of open response questions, provide MCAS practice and instruction in vocabulary, and refer students to after school tutorial programs. Teachers in interviews and focus groups communicated that they are working hard on these improvement efforts, especially the MCAS remediation plan, the use of open response questions and the use of rubrics to score them.

While interviews and documents show overall support for improvement initiatives, teachers expressed the need for more inservice. Some stated that they had not received any training prior to implementing the new Connected Math series, although Connected Math workshops were included in the Summer 2000 list of offerings. In the faculty survey, 51% of faculty identified the need for professional development in the area of effective instructional methods for diverse learners. However, the panel did not see evidence of this topic being addressed in professional development opportunities provided by the district.

C. Is the school receiving adequate guidance and support from the district leadership?

The Superintendent is well informed about the needs of the Williams North Middle School. The district provided training to administrators on the process of improvement planning, and they are developing a template to be used next year. The Superintendent is also in the process of reorganizing both middle schools, which share the same building, so that they become one school. Her intent is to hire one principal who is a skilled instructional leader, two assistant principals, one of whom will focus on academics and one on day-to-day operations, one coordinator of special education, and two deans of students who will deal with concerns such as discipline and attendance. In so doing, she is looking to provide better consistency and scheduling flexibility and to eliminate inequities that currently exist between the two schools.

When the Superintendent first assumed her position in February 2000, she asked the middle school principals what resources they needed. She also appointed a lead principal for secondary education (the high school principal) who acts as a resource to the middle school principals. The Title I, Bilingual, and Special Education Directors have observed in middle school classrooms, at the Superintendent's request, to focus on instruction and communicate with principals about any identified issues, such as the need to use grade level materials in bilingual classrooms and the need to make accommodations in instruction.

The Superintendent indicated that money is available for any professional development needed. The district's comprehensive professional development plan includes one-half tuition reimbursement on up to three courses, a listing of over 120 courses and workshops (25 specifically for middle school), and provision for the utilization of both lead teachers and mentor teachers who serve as means of support to classroom teachers. However, the district provides no early release time.

The Williams North has two lead teachers-one in Literacy and one in Math-who, in the principals' words, demonstrate, co-teach, observe, and release. He expressed the need for someone in charge of all four content areas. Middle school curriculum committees for math, science, social studies, and English language arts meet weekly or biweekly to align curriculum with the frameworks. The two middle school principals chair these committees.

Faculty survey respondents and teachers interviewed during the visit indicated their concern about large class sizes which prohibit teachers from giving individual attention to those who need it. One teacher has had a class of anywhere from 28 to 31 students. Teachers also voiced their concern about the low teacher salaries compared to surrounding communities, and the principal indicated the difficulty he has had keeping teachers as they leave for jobs in other districts.

While there is evidence of district guidance and support, panel members also sensed an underlying turmoil within the district. A concern raised by teachers was the inconsistency in philosophy and methodology due to the constant change in superintendents. (They have had six in thirteen years.) District administration also expressed differences in philosophy held by various administrators. When panel members asked questions about the relationship between the district and Boston University, there was a universal response that BU has brought much to the district but that much time was lost as the two organizations built trust in each other. One faculty survey respondent summarized the state of the district as follows: "the glue of a strategy that provides constancy of purpose for creating an environment of continuous change along with providing the basic problem solving skills to identify break-through issues is lacking from top to bottom." The panel agrees that the district will have difficulty keeping its focus on improving instruction and student performance without a measure of district level stability that can provide the vision and constancy of purpose to move the district forward in an ongoing, systematic way.

Conclusion

The Williams North Middle School has developed a plan for improving student achievement that includes numerous initiatives, strategies and action steps. However, they are based on a general analysis of data and program information to determine causal factors, and they lack clear statements of problems, desired outcomes, detailed action plans, methods of evaluation, and meaningful timelines. The faculty and administration are committed to improving curriculum and instruction with the goal of in turn improving student performance, but they will face yet another major transition in the fall due to the planned reorganization of the middle school. This will have a significant impact on all aspects of the school, and it will be necessary to stay focused on teaching and learning if student achievement is to improve.

Appendix A

Team Members

Margarita Laboy, Williams North Panel Coordinator, Massachusetts Department of Elementary and Secondary Education, Office of Accountability and Targeted Assistance.

Dr. Susan Rubel, Williams North Panel Chairperson, Coordinator of the International Network of Principals' Centers at Harvard University, consultant for SchoolWorks, Beverly, MA.

David Deschamps, Classroom teacher, Hardwick Elementary School, Gilbertville, MA

Dr. W. Spencer Mullin, Principal, Nettle Middle School Haverhill, MA

Appendix B

Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Please inform all school faculty and students that Review Panel members will be visiting a cross-section of classrooms during the site visit. The selection of classrooms will be determined mutually by the Panel Review Coordinator and the Principal using the staff directory information provided by the school. All faculty members are asked to be prepared to accommodate a visitor on the morning of the site visit. Panel members will make every effort to minimize the disruption of planned classroom activities.