Follow-up School Panel Review of The Williams Middle School, Chelsea, Massachusetts
Introduction
The Williams Middle School was one of twelve middle schools referred for panel review in the spring of 2001 as a result of critically low levels of student performance on state MCAS assessments in 1998 and declining MCAS results in 1999-2000. A panel review was conducted in March of 2001. At that time the panel found that the Williams Middle School did not have a formal plan for improvement and that improvement activities lacked sufficient focus and coordination to serve as a guide for improved student results.
Upon consideration of the panel findings, the commissioner deferred action on the determination of under-performance for a period of six months, and provided a $25,000 grant to support planning and school improvement efforts at that time. The principal and a planning team from the Williams Middle School participated in these facilitated work sessions during June, August and September, at which department technical assistance staff and data analysts guided the school's planning team through an inquiry-based process designed to help them develop a sound plan for improving student performance at their school. At the conclusion of the six-month deferral period, a follow-up review was conducted at the school by a four-member review panel on October 25-26, 2001.
The follow-up report contained some positive news indicating that the school had made important progress in beginning to use data to identify gaps in student performance and plan improvement efforts. However, the follow-up panel found that the school improvement plan did not reflect an adequate understanding of the gaps in curriculum and instruction related to student learning, nor did it set forth a coherent plan to address priority areas. The result of these findings was to give the school six more months in order to continue its improvement planning work.
On October 9, 2002 a three member review team returned to conduct the second follow-up review.
The Scope of the Second Follow-up Review Process
The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:
- Does the school have a sound plan for improving student performance?
- Are the conditions in place for the successful implementation of the school's improvement plan?
The panel's responses to the two key questions that defined the scope of their review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to them. A list of panel members who participated in the Williams Middle School review is provided in Appendix A. An illustrative schedule of the panel's activities is provided in Appendix B.
The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Williams Middle School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist, or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.
Panel Reponses to the Key Questions
KEY QUESTION 1: does the School Have a Sound Plan for Improving Student Performance?
Within the framework of the second leadership change in as many years and a series of initiatives in the early stages of implementation, the Williams Middle School has promulgated prioritized Action Plans in English Language Arts and Math designed to improve student performance. They have articulated causal factors for poor student achievement and corresponding problem statements and strategies to address these issues. The school's analysis of multiple data sources, coordination of information and attempts to build staff endorsement satisfy concerns mentioned in previous reports. The Panel believed that the plan would have been enhanced by a more expansive discussion of student performance determinants, with particular attention to the achievement of specific student populations.
A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?
There is evidence that the Williams School utilized appropriate data analysis procedures to determine the gaps in student performance and identify reasons for those gaps. Results of the 2001 Stanford Diagnostic, MCAS and district grade eight exit exams were employed to provide a performance profile in English Language Arts in grades 6-8. Similarly, MCAS, California Achievement Tests as well as school projects and tests informed conclusions about student math performance.
The school's analysis of student achievement considered a variety of school factors and was conducted at several levels. Action Teams were organized with district support (consultants and released time for teachers) from February through June 2002. The Department of Elementary and Secondary Education improvement planning procedure was followed to identify underlying problems, causes and remedies. Analysis of test data was accompanied by an examination of the school's Mathematics and English Language Arts curricula in both content and coverage. The efficacy of the literacy instructional programs was also evaluated. A multiple year review (1998-2001) of MCAS Math results revealed significant deficits in the areas of proportional reasoning and open response questions. The item analysis of English Language Arts test results disclosed difficulties in students' abilities to identify unknown words, self-monitor reading comprehension and organize/synthesize information when composing written responses to reading passages.
Through faculty interviews and an examination of new curriculum documents, the Panel noted a narrowing of focus in the ELA Action Plan and the development of an organized vocabulary strategy, the adoption of a specific reading comprehension program, and the school-wide implementation of the John Collins Process Writing Program. As a result, the school was better positioned to make staffing, instructional, organizational, and assessment changes that provided greater structure and specificity to the learning environment. A review of the Connected Math Program (CMP) and MCAS results produced a realignment of unit coverage, altered time allocations and increased emphasis on performance- based math writing. An outcome of this assessment was a professional development series for teachers designed to help students with open response skills.
No recent staff surveys were administered to reflect teacher perspectives on areas of need. Priorities from a 2001 survey did surface in the plan's initiatives. The prominent initiatives recommend strengthening writing skills, open response work, content-specific professional development, MCAS vocabulary, instructional support for diverse learners, increased teamwork opportunities for staff and provisions for small group instruction.
In formulating its curriculum, program, service and assessment initiatives, the school made use of the data it gathered and analyzed, but some key limitations were noted. Disaggregated MCAS test data for 2002 had not yet been released at the time of the visit. There was no disaggregated information available for 2001. The school's only sixth grade cluster provides service primarily for special needs and bilingual students. One of its seventh/eighth grade clusters is a bilingual, sheltered and ESL/TBE team. Examining the accomplishments of these sub groups is vital, since over sixty percent of the pupils come from homes where English is not the first language. In the past, the performance of special needs students at the school was substantially below district and state averages. As current MCAS reports arrive, it is imperative for the school to evaluate this data to determine if the nature or level of service should be revisited for these student populations and to track the effect of the increased inclusion model on individual student achievement.
The administrative team is committed to the concept of providing equitable access to the curriculum. This stance has affected program and service decisions for special needs, Title I and bilingual students. Generally, teachers are using grade level material with the charge to offer an academic challenge to all students. The school's Action Teams ascertained that the programs for special needs, Title I and bilingual students had been scattered and inefficient. Greater care was taken this year in building the schedule to allow most school clusters to be served by their own Title I, ESL and special needs teachers. This has substantially reduced pullout service and has further operationalized the inclusion policy to which the school is dedicated. However, without a systematic analysis of the performance of these students, the impact of grouping and instructional practice will be difficult to evaluate. Additionally, although support service for grade clusters seems adequate, a review of the professional development menu for the school and district indicated no course work for teaching diverse learners at the middle school level.
The format used to present student test performance data was exclusively statistical without a discussion of the linkage between test data and the school's selected improvement initiatives. Readers of the improvement documents would benefit from further explanation and rationale for the school's choices. Future school improvement documents would be enriched by interpretive, summary statements, constituting a bridge between data findings and improvement initiatives.
B. Does the plan set out specific improvement objectives that are grounded in the school's analysis of the reasons for poor student performance?
The reasons for poor student performance in Math and ELA are identified in abbreviated notations attached to the major problem statements. There was no description of other factors that might have been discussed in the analysis of student achievement gaps. For example, school Action Teams stated that students have difficulty organizing and synthesizing information as they craft written answers to reading passages. In the list of stated explanations for this weakness, there was no indication that the actual teaching of this skill was considered. A more complete narrative of issues that were reviewed would have been instructive to the Panel. Nevertheless, the conclusions are drawn from numerous data sources, articulate the overriding problems in each subject and make the connections between aspects of under-performance, improvement strategies and anticipated outcomes. The stated instructional priorities and curriculum revisions created a connection between the new reading comprehension, writing and vocabulary programs. Subject based learning outcomes are outlined in the revised curricula. The curriculum and administrative changes enacted this year make the improvement objectives more specific, measurable, reasonable and achievable. The Panel felt that the school has established the means to define the tie-in between learning and performance objectives, as they are embedded in the curriculum and can be incorporated into a more inclusive Whole School Improvement Plan next year.
Student improvement objectives, timelines and implementation activities have been made more specific. The plans set out short term benchmark activities to monitor student performance targets. The school will administer quarterly literacy and math assessments using a September baseline score to track student progress in November, February and April. The instrument will use MCAS and school curriculum -generated material. Quarterly improvement goals have been created based on moving a given percent of students out of the "warning" and "needs improvement" categories to "needs improvement" and "proficient" levels. The assessments will form the basis for referrals or service from Title I, student intervention (guidance, social work), special needs consideration, further diagnostic testing or after school academic support. Curriculum teams will consult following each quarterly assessment as a program evaluation measure. At the conclusion of the first quarter this year, the Stanford Reading Diagnostic will be given to chart correspondence between that instrument and school-produced benchmarks. This links external standardized assessment to local student achievement measures.
Identified reasons for poor student performance shaped many of the specific improvement strategies selected at the school. For example, the work of the Action Team revealed the need and commitment to provide training to all staff in the teaching of vocabulary, reading comprehension, writing strategy and rubric development. It was determined that all teachers would benefit from exposure to math content and practice particularly in the topic areas of ways to represent data and proportional reasoning. Interviews with individual teachers, curriculum committee representatives, lead teachers and administrators suggest early benefits to cross-discipline professional development, citing the inherent value in supplying full faculty training in math problem solving techniques and critical thinking processes.
The strategy to have all staff acquire and employ literacy and math training has impacted the school improvement process. It has driven the school's academic objectives, the professional development agenda as well as the schedule and deployment of personnel. The full staff received forty hours of professional development during the summer and fall in vocabulary development, reading comprehension, writing/open response and rubric process, multiple representations of data, CMP and proportional reasoning. During the spring of 2002, all staff participated in John Collins training. This preparation, and the ongoing cross discipline instruction, has cultivated a common language in Math and ELA. Several teachers credited the learning climate shift already in evidence this year as creating a changed reading culture and establishing a philosophy of how to respond to questions.
Ancillary changes, not contained in the Action Plans but associated with improvement objectives, appear to have had a positive effect on the social environment during the first two months of school. Teachers, parents and the School Council pointed to improved discipline conditions. September suspensions were at their lowest level in many years. Classrooms within a cluster are now in closer proximity to one another, limiting unnecessary hall movement and increasing instructional time. The closer connection to teachers is also a function of the extra time students spend with their homeroom teacher during the new Advisory and Literacy blocks. Common planning times, during which clusters meet at least once a week, are opportunities to discuss matters that relate to student discipline, academic need, curriculum, instructional practice, or full team meetings with parents. Teachers report that students are aware that collaborative planning or dialogue about their progress is now taking place. While discipline has traditionally been a major obstacle to learning, the plan contained only a brief reference to a new Code of Discipline and no specificity with respect to goals, strategies or ways to evaluate improvement.
C. In order to accomplish each improvement objective, does the plan specify strategies which appear likely to lead to improved student results?
There are a number of indicators that offer the promise of improved student learning. Last spring there was a clear identification of learning barriers and proposed corrective initiatives. The work was carried out through the collaborative planning of teachers, administrators, consultants, the Department of Elementary and Secondary Education and curriculum committees. Action Plans have been bolstered by revised, aligned curriculum documents that tie learning strands to specific resources and lessons, a scope and sequence for all subjects, acquisition of many new texts and materials, and the introduction of a schedule to regularly assess student performance.
Instructional time and consequent changes in the school's master schedule reflect increased attention to math and literacy. The schedule includes a twenty- five minute Advisory Period at the start of each day where Math Problems of the Week are solved and interdisciplinary vocabulary exercises are conducted by the homeroom teacher for each class. Problem of the Week packets are prepared each week by the math department and serve as a directed guide for all subject teachers to lead their students through the problem solving process. All students have a sixty -minute literacy block in the middle of the day that involves critical reading work with school anthologies and an array of writing challenges. The Connected Math Program has been reconfigured by data analysis to encourage teachers to think in terms of monthly plans, create and apply an essential math vocabulary, and elevate coverage of those topics that have emerged as chronic program weaknesses.
The action steps for improved Math and ELA achievement largely govern the use of support staff. Title I reading teachers work during the Literacy block with science, history and math teachers engaged in ELA work with their students. This block supplements the students' regular ELA period where mechanics and grammar are taught. Literacy initiatives are enhanced by six Title I teachers serving specific grade clusters as co-teachers, teacher trainers, remediation specialists, curriculum resources and as a source of feedback for both teacher performance and administrative evaluation of the literacy program. Two full-time math lead teachers perform similar roles and also do demonstration lessons, coordinate the training process for quarterly assessments, organize the after school academic support programs and handle professional development activities. While these role descriptions are well conceived, there was no indication that plans are in place to review the efficacy of these functions either on a periodic or long-term basis.
D. Are the school's written improvement planning document (s) clear and specific enough to guide their implementation of planned improvement initiatives?
The basic components of the Action Plans are specific and have the capacity to guide the implementation of the improvement initiatives. The draft of the plan preceded the significant summer curriculum work, obligatory school-wide professional development in literacy and math and decisions to include the Advisory and Literacy slots in the schedule. The principal's choice to add lead teachers in science, history, ESL/TBE and special needs and enhance inclusion within grade level teams also followed adoption of the plan and served to strengthen the planning documents. These steps, combined with increasing staff optimism about the first two months under a new leadership team, could certainly provide for an expanded, more explicit version of the improvement plan initiatives, even at this early date. Timelines seem legitimate. Staff appears satisfied with the resources in place and the plans to use them.
Several new procedures instituted this year also support elements of the Action Plans and promote the professional culture of the school. A new lesson plan template calls for notations of stated objectives, learning strands, vocabulary and assessment. The process of teacher goal setting is increasingly aligned with school objectives. The submission of weekly common planning time cluster reports is a self-evaluation tool for teams and a source of information and insight to administration. The completion of Student Success Plan profiles for struggling learners chronicles progress for parents and faculty.
The roles of key personnel appear clear, understood by faculty and presented in a logical, coordinated way. For example, the part-time literacy consultant, assistant principal for curriculum and instruction, Title I reading teachers, and curriculum committee are all involved in quarterly benchmark assessments and follow-up analysis of findings. Properly managed, this complex of functions can be an effective vehicle to inform curriculum and instruction.
The previous School Improvement Plan dealt with an issue about wide discrepancies between subject grades assigned to students and their MCAS scores. It was a matter covered in some depth because there was a major mismatch between the two measures of student performance. Our Panel was unable to learn from those interviewed how this problem was resolved- if there was an exploration of or change in grading policy. This highlights the need or the school to actively address relevant questions that emerge from an examination of data.
While the student performance benchmarks are frequent and linked to improvement objectives, the Panel believes that the school should be able to evaluate progress in behavioral terms, within the plan itself. This could ultimately take the form of exit skills for students- what they should know and be able to do at the conclusion of each grade. A more performance -based format for teachers could include an identification of instructional expectations, or a delineation of what effective practice looks like. All teachers participated in workshops to develop and use rubrics to promote students' ability to self evaluate and internalize standards for exemplary responses. The panel received conflicting information as to whether school-wide, standardized rubrics (by subject) are to be employed or if the process is a personalized one for each teacher.
E. Was the School Improvement Plan developed through a process that will support its successful implementation?
A substantial, representative cross section of faculty was involved in the Williams improvement process. The planning initially engaged approximately one half of the staff of over 100 teachers on the Action Teams. The group was comprised of grade, subject, bilingual, special needs, support and administrative personnel, a literacy consultant and strategic planner. Curriculum committees, about one third of the staff, worked throughout the summer to revise curriculum, to parallel documents with state standards and to couple learning objectives to the variety of texts and supplementary materials. The principal indicated that buy-in to the plan and its implementation is demonstrated by a continuing role among a cadre of teachers who taught summer workshops or have assumed a leadership role in either curriculum or assessment work.
The Panel found staff interviewed to be conversant with the plan's content and equipped with a baseline of literacy training and a familiarity with math concepts. The change in this year's schedule has added time to literacy and math activities at the expense of other disciplines. While the loss of some instructional time was mentioned by non ELA staff, there seemed to exist a general understanding that the analysis of data underscored the importance of literacy to learning.
Although there are independent measures of program and instructional effectiveness, there is no systematic evaluation component embedded within the plan. The principal and assistants have been making classroom visitations and providing written feedback to teachers. Progress benchmarks are in place and common planning time reports are issued by clusters on a weekly basis. The literacy consultant, lead teachers, Title I staff and myriad of teacher groups will have reflective input to the evolution and monitoring of initiatives. The elements for an ordered assessment approach exist, but the school's challenge is to construct a methodical description of how the systems involved in the evaluation of instructional quality might be pulled together.
KEY QUESTION 2: Are the Conditions in Place for the Successful Implementation of the Improvement Plans?
The evidence assembled by the Panel indicates that the conditions are present for carrying out the school's major improvement initiatives. The new administrative leadership team has used professional development to provide clarity to the school's improvement objectives and assembled the resources and organization to support the efforts of the staff. Teachers interviewed described satisfaction with the curriculum adjustments, instructional direction and expressed confidence in the leadership team.
A. Does the school have effective leadership and sound management?
The Panel Review Team heard broad stakeholder expressions of respect, trust and confidence in the new principal. A veteran principal in the district, she was asked by the Superintendent to provide direction and strong leadership at a critical juncture. The role of the School Council was gradually expanded, the work of the Action Teams coordinated, and this September she hired new assistant principals for curriculum/instruction and special education and reassigned a third assistant to handle operations. The Panel observed the duties and goals of each assistant principal to be clearly defined and well tailored to their previous experience. The assistants conveyed a sense of energy, professionalism and vision, each articulating the importance of an integrated, inclusive approach to improving student achievement. They were also able to articulate how the work in their domain influenced the larger change effort. The Assistant Principal for Operations had already experienced success with the schedule and Code of Discipline. The Assistant Principal for Curriculum and Instruction is overseeing the continuing revision process that represents a more complete, connected and relevant series of documents.
Teacher and parent interviews revealed a climate of increased expectations based on a few critical beliefs outlined by the principal and implemented through a number of key administrative actions. The principal talks about opportunities and challenges, not obstacles, refers to the power of having students accountable to specific adults, and the need for full-staff literacy skills as a generic imperative. Project Challenge, a high level math program that began in the elementary schools four years ago is being sustained at Williams. Every seventh and eighth grade cluster has one class organized around that initiative. She accentuated the importance of making assessment a routine part of the school day for teachers and students and the systematic mentoring, supervision and feedback of teacher performance. Although these initiatives are not yet fully developed, Panel interviews with teachers and the principal recognize the framework for implementation. The principal reported that the hiring process this past spring and summer was somewhat difficult because she was intent on attracting content area specialists with some experience. The mentoring and supervision functions will be particularly critical because of the large number of teachers with non-professional status.
Based on the exigencies of the school, many of the management decisions were centralized. The Panel understood the more immediate need to set a course for the school and pull the staff together in the goal setting process. Teacher groups have coalesced around an assortment of projects and should now be ready to take on an increased governance role in the school. The School Council noted a greater consultative relationship with the current principal. The Assistant Principal for Operations indicated that in doing next year's schedule there will be early collaboration with the teaching staff. After several recent changes in administration, a merger and school reorganization in 2001, the staff seems poised to exercise some shared decision -making responsibility. There are ingredients for a site based management approach to take hold, but they have not yet been actualized.
B. Is there evidence that the school's faculty supports the planned improvement efforts?
The Panel sensed the presence of a common language and shared purpose among the staff at Williams. Improvement objectives are understood and frequent reinforcement occurs through daily curriculum work, professional preparation and various communication channels during an average day or week (cluster, department, grade, staff meetings). The data analysis procedure exposed faculty to many ways of looking at student performance. The finding that so many students failed to even attempt open response questions confirmed teachers' perceptions in this area, but also allowed them to appreciate the magnitude of the problem.
The faculty was meaningfully involved in the adoption, training and implementation of the plan, and so its acceptance appeared widespread among those seen by the Panel. In discussions with our Review Team, teachers remarked that structural changes and elevated expectations are evident and felt by students and parents. They referred to the school's persistence in sustaining contact with the parents of children requiring after school academic support, until they secure permission or refusal for service. The Student Success Plan document keeps parents informed of the academic status of students experiencing difficulty throughout the year. Although this is one of many additional tasks taken on this year, teachers approve of its purpose.
While the number of classroom visitations conducted by the Panel was limited, good communication between teachers and students was observed, as were examples of conversational teaching and project-based activities. The student interview sample was small and pupils were at different performance levels. They expressed the view that they were regularly challenged and that the classroom norm is active learning with lots of pupil -teacher interaction.
C. Is the school receiving adequate receiving adequate guidance and support from the district leadership?
While the school district provided the resources needed to launch the planning effort last spring, the Panel found mixed evidence of the district maintaining an active, continuing role in the formulation, guidance or monitoring of the school's improvement initiatives. The faculty strongly commended the professional development support supplied by the district. Professional development decisions are generated through staff needs surveys and are designed to parallel key goals underway in the schools. The release of teachers to engage in the Action Team process, the availability of consultants and the funding for mandatory training were vital to completing the planning work. Staffing at the school is adequate and appropriate.
The last Panel report indicated a lack of clear district instructional priorities or strategies to evaluate programmatic effectiveness. That judgment is verified in this visit as well. Although the district is now engaged in a five- year strategic planning process, the Review Panel was unable to identify district level academic initiatives or an improvement plan in the material provided to us. Faculty interviewed did not speak to a visible district presence or guiding function at the school. There was no discussion or documentation that the district participated in data analysis, curriculum revision or alignment, meetings with curriculum committees or school clusters.
Conclusion
The Action Plans for improved student performance are focused in literacy and math and map a direction for achievement based on collective test data and curriculum analysis. A coordinated group process identified the root causes of student performance gaps and proposed improvement objectives. Early indications suggest that the new principal has a good sense of what needs to be done to improve student achievement and the school environment. She has used personnel resourcefully, assembled a good administrative team, applied staff professional development to the identified areas of weakness and produced a schedule that specifically targets a range of real priorities.
The school's recent history of leadership changes, reorganization and lack of strong district oversight of curriculum compliance and program evaluation, suggests that this plan places the school on the right track but is still in a formative stage. It forms the basis for a more exhaustive Whole School Improvement Plan next year that is framed on the experience of this year's initiatives. While it is not covered thoroughly in the current plan, the Panel reiterates the importance of establishing procedures and standards to evaluate the impact of realigned curricula, instructional methodology, assessment processes and professional development. Because they are issues attendant to improved student learning and are being managed as part of this year's work, student attendance and discipline, staff communication and morale as well as parent- community engagement should be formally integrated into the improvement design. This effort will require the active and consistent support and evaluation of the school district. As the district assiduously monitors the improvement process at the school, it might seek standards to gauge the degree to which the faculty has internalized the improvement initiatives.
Denise Delorey, Department of Elementary and Secondary Education. Accountability and Targeted Assistance
Joyce G. Harrington, Ph.D. Assistant Superintendent for Curriculum and Instruction, Bourne Public Schools
Nick Feldman, Consultant, Schoolworks
last updated: November 22, 2002
|
State Government · State Services 
