Assessment/Accountability
School and District Accountability and Assistance

Follow-up School Panel Review Report of the Consentino Middle School
Haverhill Public Schools

Introduction

The Consentino Middle School was one of twelve middle schools referred for panel review in the spring of 2001 as a result of critically low levels of student performance on state MCAS assessments in 1998 and declining MCAS results in 1999-2000. A panel review was conducted in March 2001. At that time the panel found that the Consentino Middle School did not have a formal plan for improvement and that the improvement activities lacked sufficient focus and coordination to serve as a guide for improved student results.

Upon consideration of the panel findings, the commissioner deferred action on the determination of underperformance for a period of six months, and provided a $25,000 grant to support planning and school improvement efforts during that time. The principal and a planning team from the Consentino Middle School participated in these facilitated work sessions during June, August and September, at which department technical assistance staff and data analysts guided the school's planning team through an inquiry-based process designed to help them develop a sound plan for improving student performance at their school. At the conclusion of the six-month deferral period, a follow-up review was conducted at the school by a four-member review panel on November 1, 2001.

The follow-up report contained some positive news indicating that the school had made important progress in beginning to use data to identify gaps in student performance and plan improvement efforts. However, the follow-up panel found that the school improvement plan did not reflect an adequate understanding of the gaps in curriculum and instruction related to student learning, nor did it set forth a coherent plan to address priority areas. The result of these findings was to give the school six more months in order to continue its improvement planning work.

On May 28-29, 2002 a three member panel review team returned to conduct the second follow-up review.

The Scope of the Second Follow-up Review Process

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan(s)?

The panel's responses to the two key questions that defined the scope of their review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to them. A list of panel members who participated in the Consentino Middle School review is provided in Appendix A. An illustrative schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Consentino Middle School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist, or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Consentino Middle School Profile

The Consentino Middle School enrolled 734 students in grades six through eight as of fall 2001. Thirty-nine percent of the students were eligible for free or reduced-price lunch, and 15 percent spoke a first language other than English. The school identified three percent of its students as Limited English Proficient. The student body is identified as predominantly white (69%) with sizable Hispanic (23%), African-American (5%), and Asian (3%) populations.

Consentino Middle School MCAS Results

Cycle 1 Results

Overall MCAS results showed a decline in 1999 and in 2000 were nearly identical to baseline scores. In comparing the 1998 baseline score to the 1999 and 2000 average, scores were flat in English Language Arts, decreased by two points in Mathematics, and increased by one point in Science & Technology. In 2000, 26 percent scored in the Warning performance level in English Language Arts, 70 percent scored in Warning in Mathematics, and 58 percent scored in Warning in Science & Technology.

2001 Results

A= Advanced
P= Proficient
NI= Needs Improvement
W= Warning (formerly Failing)

Panel Reponses to the Key Questions

KEY QUESTION 1: does the School Have a Sound Plan for Improving Student Performance?

Yes. As a result of a year long school improvement planning process, the Consentino Middle School has produced a school improvement planning document focused on three priority areas with goals, benchmarks, timelines, and responsible parties that could be identified as a "sound plan."

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

Yes. With support from the Department of Elementary and Secondary Education and the district, the school has continued to use MCAS data in planning school improvement efforts. As indicated in the previous follow-up panel review report, during the summer of 2001 the school looked at grade 8 MCAS scores in ELA and Math (2000), grade 7 ITBS in reading, language arts and math (1999-2001), grade 6, 7, and 8 district writing assessment results, and grade 6 end of year math assessment results to identify gaps in student achievement. Through this process, the school identified five improvement areas, three of which were identified for priorities for the 2001-2002 school year: English language arts, math and attendance. The school developed turnaround plans for each of these areas under the guidance of the Department of Elementary and Secondary Education technical assistance staff.

Since the summer training, the Haverhill School District has provided data analysis support. In October 2001, the district provided the school with data from the 2001 MCAS. The district generated reports that included dissagregated data by race/ethnicity, gender, students with disabilities, English language learners, and students on free/reduced lunch. The analysis revealed that overall Consentino students have improved performance in English language arts. Results from the 2001 Grade 8 MCAS show that the number of students in the Warning performance level dropped from 26% to 12%. The number of students in the Proficient performance level increased from 35% to 47%. While overall performance is improving in English language arts, the warning rates for Latino, special education and students who receive free and reduced price lunch have increased.

On the Math 2001 grade 8 MCAS, Consentino students performed poorly. Although there was a 12% reduction in the percentage of students scoring in the Warning performance level, school leaders acknowledge there is much room for improvement. Item analysis shows that students performed poorly on questions related to number sense, operations, geometry, measurement and on open response format questions. The same subgroups of students who did poorly on the English language arts test, namely Latino, special education and those who qualify for free and reduced price lunch, did poorly on the math assessment as well.

In addition to analyzing MCAS scores, the school has looked at results on the Iowa Test of Basic Skills (ITBS) for 1999-2001, the Haverhill district writing assessment in grades 6, 7 and 8 for 2001 and grade 7 for 2002. Among the many findings, the Haverhill writing assessments reveal that students in grades 7 and 8 showed an increase in the percentage reading proficiently, although according to district data analysis Consentino scores are below the district averages for 1999, 2000 and 2001.

Teachers and administrators are aware of the different performance levels among subgroups of students. However, they are less articulate about the specific causes. For example, when asked about the causes of low scores for English language learners on the ELA test, teachers and administrators responded with vague notions about the students such as, "students don't have enough vocabulary" or "language is a barrier." In math, administrators noted a more rounded analysis of the causes of poor student performance and cited the difficulty of finding certified math teachers, a shift to a new math textbook, and a question of appropriate instruction for all students. According to the district math coordinator and school administrators, the school is addressing these causes by aggressively recruiting certified math teachers and providing some training for teachers in instructional strategies.

Interviews with teachers revealed their increased awareness of the importance of data for decision making. Teachers talked about reviewing school assessment data during monthly staff meetings and team meetings and how they track progress in their classrooms. One teacher notes, "I am more aware of the differences of how students receive information."

Finally, student attendance, the third priority area, has shown improvement. During the past five years, there have been steady gains in student daily attendance. For example, daily attendance went from 91.2% in 1997-1998 to 94.3% in 2001-2002. While this is promising, attendance rates for the same groups of students scoring lower on English and math assessments are not improving. Teachers and staff cite a number of causes for this including the need for better outreach to certain groups of parents.

Yes. Action plans in each of the three areas ---English language arts, math and student attendance--- set out broad student improvement objectives and strategic objectives. For example, in English Language arts, a student improvement objectives states, "Improve student performance levels on all portions of MCAS testing in grades 6, 7, and 8." The strategic objective is, "students will complete more writing in all curriculum areas. They will complete a minimum of 40 Type III, Type IV pieces."

The student improvement objectives and strategic objectives are focused on teaching and learning and are reasonable and measurable. However, they are very broad and general. For example, an ELA objective states, "Students will increase general knowledge and use of vocabulary." The panel visiting the school remains concerned that general objectives may not be specific enough to target the needs of English language learners and Special Education students, two subgroups of students who are not improving at a rate comparable to the general student population.

C. In order to accomplish each improvement objective, does the plan specify strategies which appear likely to lead to improved student results?

Some of the strategies in the plan have already resulted in improvements. According to the school's progress report, all of the strategies in the school improvement plan for increasing student attendance, such as adopting Nordex (an administrative software that enables student attendance data to be collected and analyzed), conducting a school-wide truancy program, adding a middle school attendance counselor, and creating a nightly call list to contact homes of students who are absent, have been implemented. As a result attendance has increased one full percent over last year.

Strategies for improving literacy include the John Collins writing program (a comprehensive writing across the curriculum program), the purchase of a remedial reading program called READ 180, and increased MCAS vocabulary development across content areas.

Currently, the John Collins writing program is being used across content areas. In interviews, teachers across subject areas mentioned this writing program as a unifying instructional approach for the school. Teachers brought writing folders and showed panel review members examples of student progress. When asked how the program is working, teachers overwhelmingly cited evidence for increased student success. However, teachers were less able to articulate how the program would specifically target the needs of English language learners other than citing that English language learners will be expected to write more.

READ 180 is a remedial reading program has been fully implemented with targeted students in grade 6. Most of the students identified for the program are SPED students and English language learners. While READ 180 advertises its effectiveness for all students, the school does not have data as yet to verify its impact on student progress. According to the school improvement plan, the school will have a full implementation schedule and plans for monitoring the program's impact by September 2002.

Math is an area of concern to administrators at the school. There are a number of strategies in the school improvement plan that are targeted to improve math results. Principally, the district has recently adopted a textbook that is aligned to the mathematics Curriculum Frameworks. This new textbook represents a shift in the way that teachers have taught math in the past and will require professional development and classroom support. Strategies in the school improvement plan targeted to improving math include: administering diagnostic pre and post tests and end-of unit assessments, participating in the Middle School Math Initiative, which includes a full-time math teacher who provides ongoing training in lesson planning and assessment, continuing the use of math folders to store samples of student work, and teaching geometry and measurement earlier in the school year. In addition, the district is in the process of realigning the math curriculum maps. According to the school improvement plan, realignment will take place during the summer of 2002, and teachers will review these maps during their monthly curriculum meetings. In interviews, administrators cited the curriculum improvements as an important component in their effort to improve student progress.

While there is some expertise in the building regarding the education of English language learners, a sub-group that is identified as chronically underperforming in the school, expertise in supporting this group of students is not sufficiently widespread and coordinated. Teachers are aware of the gaps in English language student performance and meet in grade level teams daily to work on instruction. However, in order to raise achievement of these students, more building and district level expertise is needed. When asked where teachers access specific information on English language learners, teachers responded that they sometimes seek the advice of the ESL teacher. They generally rely on their own knowledge to make instructional decisions for these students.

D. Are the school's written improvement planning document(s) clear and specific enough to guide their implementation of planned improvement initiatives?

The school improvement planning document is in three sections: English language arts, math and student attendance. Each section contains data analysis, data sources, a problem statement, suggested root causes, and action plans. The action plans include student improvement objectives, strategic objectives, action steps, responsible parties, resources and timelines. In addition, each strategic objective has a monitoring plan that includes the targeted skill or concept, targeted student group, benchmarks, assessment tools and dates, and responsible parties.

The three plans --- English language arts, math and student attendance --- are detailed. Although it is not explicitly stated in the plans, they are integrated primarily through the writing across the curriculum effort, professional development addressing differentiated instruction, and the school-wide focus on MCAS vocabulary and open response questions. The next step in the planning process is for the school to be more explicit in how the plans are integrated.

E. Was the School Improvement Plan developed through a process that will support its successful implementation?

Yes. The school's leadership team attended the Turn Around training retreats sponsored by the Department of Elementary and Secondary Education in June, August and September. Thirty staff members worked at the school during the summer to implement the Turn Around process. Participants worked together to analyze data, prioritize concerns and begin to develop the school improvement plan in teams. Content area teams in math and English met separately during July and August to complete drafts of the plans. In October 2001, draft plans were distributed to the entire staff for discussion and feedback.

During the recent 2001-2002 school year, school improvement plans were discussed within the meeting structures of the school. Grade level teams meet daily and frequently to look at the student writing progress. During monthly staff meetings, broader school improvement planning and monitoring issues are discussed. For example, during the last all-staff meeting, staff discussed progress made on the school improvement plan and prepared for the panel review visit. All-staff meetings slated for the future focus on revisiting school data and monitoring improvement strategies. Finally during monthly content area curricular meetings, teachers discuss aspects of the curriculum and classroom practice. During interviews, teachers who were not part of the initial turn around school improvement training cited that they felt informed and part of the planning process. In addition, they were able to articulate their responsibilities in implementing the plan.

The school site council has also played a role in the school improvement process. Minutes from school site council meetings indicate discussion of the school improvement plan and the panel review process. During interviews, school site council members could discuss aspects of the plan and their support for teachers and administrators in implementing the plan.

Concurrent with the school improvement planning process, a number of staff participated in Instructional Leadership Training (ILT). Based on the need for a greater focus on improving classroom practice, administrators and volunteer teachers met together to receive training on indicators of effective teaching and to conduct classroom observations. During interviews a number of participants indicated that this ILT training had a significant impact on professional relationships and climate within the school. One administrator recommended that peer observation become a standard practice for professional development across the district.

In an effort to identify the root causes of poor student achievement and improve instruction, all staff participated in a series of workshops on differentiated instruction. During March, April, and May, 2002, staff participated in 6 two-hour sessions offered by the Massachusetts Teacher Association. In interviews, teachers reported that the training was helpful. One veteran teacher noted, "I didn't expect to learn anything new, but I did." According to the principal, this is the first step towards "promoting research-based instructional practice."

Professional development is offered through the district and is menu driven. While the ILT training and differentiated instruction emerged directly from school-based needs, there is no coherent school-wide professional development plan that goes beyond the current school year. When asked about professional development plans for the upcoming school year, administrators stated that budget cuts at the district level have left no professional development funds at the building level. Professional development is being handled district wide, and there will be an additional contracted release day (October 25) in the coming year for district-wide professional development.

KEY QUESTION 2: Are the Conditions in Place for the Successful Implementation of the Improvement Plan(s)?

Yes. The conditions for successfully implementing improvement planning strategies appear in place, although there are concerns about sustaining the current level of support to the school. There is broad staff investment in the planned strategies and change efforts are already underway.

A. Does the school have effective leadership and sound management?

The school improvement efforts and day-to-day workings of the school are managed well. The leadership team, consisting of the principal and three assistant principals, has clear roles and responsibilities in leading the school. Each assistant principal is responsible for an instructional grade level. They meet weekly after school to discuss teaching and learning and to ensure consistency of effort across grade levels. For example, the assistant principals spend two periods a day in classrooms and attend grade level team meetings. One administrator said, "We do more curriculum talk now." Panel review members noted a positive rapport among the principal and assistant principals in their numerous meetings and encounters throughout the visit. One administrator noted, "By working together we have more opportunity to do long range planning."

Although the principal makes final decisions about the school, he listens to his assistant principals and gets input from staff and the school site council. In interviews, school site council members reported their support for the administrators and felt that communication flowed freely. The majority of teachers interviewed indicated the same. During parent interviews, a few parents indicated that they were not aware of the school improvement process; however, they were aware of the MCAS test. Teachers and administrators acknowledge that more could be done to involve parents. They note that the monthly school newsletter and the student agenda books were positive steps towards improved parent communication.

B. Is there evidence that the school's faculty supports the planned improvement efforts?

Yes. There is widespread buy-in and involvement among the faculty with the school improvement planning efforts. All faculty have had input into the plan. Interviews revealed that while some staff were more articulate about the plan than others, all staff could discuss it intelligently.

Not only does the staff support the plan, there appears to be a critical mass of teachers who are enthusiastic about the plan. During focus groups, teachers were enthused about the progress that students were already making as evidenced by their writing folders. Numerous staff members noted the climate change at the school as a result of the school improvement planning process. Teachers expressed enthusiasm for the plan and confidence in its success. One teacher noted, "More people are working together now." Another noted, "Even the kids seem to be part of the process. Kids are comfortable writing."

C. Is the school receiving adequate guidance and support from the district leadership?

District leaders are well informed of the needs and progress at the Consentino. The district Curriculum Director has continued to work closely with the Department of Elementary and Secondary Education technical assistance staff and the principal of the school. District curriculum supervisors attended the ILT training and are spending more time at the school.

According to interviews and the school's progress report, the district has provided considerable support for the school. For example, the superintendent approved the hiring of the 3rd assistant principal, purchased the READ 180 program (a new math textbook series), provided substitutes so that teachers could be released for ILT training and school improvement committee work, and supported a math specialist position and a middle school attendance officer. In addition, the district continues to provide data analysis in the form of school reports. Finally the district has made a commitment to return Title I funding to the school next year that will support additional instruction in ELA and math.

There is concern within the district and school regarding proposed budget cuts. The superintendent is very supportive of the school as he sees that the district has learned from the Consentino experience. During interviews, panel review members learned how the superintendent has been paving the way for continued school committee support for the school amidst budget cuts. Citing a September 2001 document, reportedly submitted to the school committee, the superintendent outlined how ongoing support for the Consentino is part of the district's core values.

Finally, there is concern about the district's ability to support the school in raising achievement for those groups of students who are not progressing. Budget cuts are forcing the district to eliminate the bilingual education coordinator position. The director of curriculum, finishing his first year in that role, will now be taking on those duties, along with functioning as Title I director next year. The district can not afford to lose expertise in supporting the academic development of English language learners.

Conclusion

As a result of a year-long school improvement planning process, the Consentino Middle School has produced a school improvement planning document focused on three priority areas with goals, benchmarks, timelines, and responsible parties that could be identified as a "sound plan."

The conditions for successfully implementing improvement planning documents appear to be in place: There is broad staff investment in the improvement strategies, and change efforts are underway. The panel remains concerned, however, that the current level of support to the school be sustained to move the improvement process forward.

APPENDIX A: Team Members

Nancy Clair Ed.D, Consultant, Schoolworks, Beverly, MA and the Center for Applied Linguistics, Washington, D.C.

Denise Delorey, Coordinator, Massachusetts Department of Elementary and Secondary Education, Malden, MA.

Carolyn Richards, Testing & Assessment Coordinator, Lowell High School Lowell Public Schools, Lowell, MA.

APPENDIX B: Detailed Schedule for Review Panel School Site Visit