Panel Review Report Edmond Talbot Middle School, Fall River Public Schools

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low and no trend toward improved student performance is evident from MCAS data. The Edmond Talbot Middle School was one of twelve Massachusetts middle schools meeting this criterion that were selected for panel reviews in spring, 2001. The Panel Review of the Talbot Middle School was conducted on February 27, 2001.

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. 1. Does the school have a sound plan for improving student performance?
2. 2. Are the conditions in place for the successful implementation of the school's improvement plan?

The panel's responses to the two key questions that defined the scope of its review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to them. A list of panel members who participated in the Talbot school review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Talbot Middle School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist, or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Edmond P. Talbot Middle School Profile

Talbot Middle School was the second largest of Fall River's four middle schools with 768 students enrolled in 2000 in grades six through eight. In that year, just under two-thirds (63%) of the students were eligible for free or reduced-price lunch, and 43 percent spoke a first language other than English. The school identified 11 percent of its students as Limited English Proficient. The student body is identified as predominantly white (77%) with sizable Asian (10%), Hispanic (8%) and African-American (4%) populations. At the time of the Panel visit, Talbot had grown to be the largest Fall River middle school with 791 students.

In 1999 (the most recent year's data available to the panel), students missed on average 14 days of school in 1999, for a daily attendance rate of 92.1 percent. In that same year, 82 students (11%) received out-of-school suspensions, while 179 (24%) were suspended in school. No students were excluded for more than 10 days. Additionally in 1999, one student dropped out, and 13 (2%) were retained in grade.

Staffing

Talbot's students are taught by 53 full-time teachers, five long-term substitutes and a teacher's aide for an approximate average pupil to teacher ratio of 13.5 to 1. The central staff consists of Principal Bruce D. Clarke, two assistant principals, two guidance counselors and a librarian. Of the 53 teachers, 23 (43%) have attained a master's degree. Of the 13 teachers listed as math or general science teachers, none holds a bachelor's or master's degree in their subject area. Three of the 13 possess math or science certifications.

MCAS Results

MCAS results have been flat over the past three years beginning and ending the cycle with an overall average score of 218.3. In each content area scaled scores were within a point of the 1999 & 2000 average. Failure rates were similar as well, falling slightly in Mathematics, rising slightly in Science & Technology and remaining the same in English Language Arts. In the past two years, two-thirds of Talbot's students have scored in the Failing performance level in Mathematics and Science & Technology, while 29 percent scored in Failing in English Language Arts. The one positive indication from the 2000 results was the near doubling (from 8% in 1998 & 1999 to 15%) of the percentage of students scoring in Proficient or Advanced in Science & Technology.

Panel Responses To Key Questions

Key Question 1: does The School Have A Sound Plan For Improving Student Performance?

Two factors compromise the soundness of the Talbot Middle School Improvement Plan: 1) The first part of it is a district-wide plan that specifies goals and strategies for all the schools, and the initiatives specified by Talbot under each goal are not clearly linked to identified school needs. 2) The second part of the plan, which describes eight initiatives, relates directly to the principals' summary assessment of priority needs. However, the action plans are based on cursory analysis of data, are not detailed enough to clarify the action steps to be taken, do not always articulate measurement means tied to the desired outcomes, and do not outline meaningful timelines.

1A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

Action steps articulated in the school improvement plan are not reflective of a comprehensive data analysis that identifies gaps in student performance.

The school has provided demographic data as part of its school improvement plan. However, at times information is inconsistent with the demographic data provided by the district to the Department of Elementary and Secondary Education. In addition, while the data was collected, it is unclear if or how it contributed to any part of the improvement planning process.

The MCAS analysis data contained in the school improvement plan consists of very general summaries from which it would be difficult to derive specific action plans. For instance, the English Language Arts Summary indicates the rise in percentage of available points in composition development and writing prompts from 1998 to 1999; the degree to which responses on both open-ended and multiple choice questions fall behind the State and District; and the increase in the average score of students with disabilities from 1998 to 1999, including the percentage of students in the failing category in 1999 compared to the District. Due to the general nature of the analysis, it is unclear how it resulted in the accurate identification of gaps in student performance.

The summary of priority needs within the school improvement plan states that data sources other than MCAS were also reviewed, i.e. Persuasive Essay results for the district, Explore, and Iowa Test results. The School Leadership Report also indicates that other data sources were used to inform school improvement planning, i.e. Terra Nova test results and the quarterly Benchmark testing in Math, Language Arts, Science and Social Studies. However, the Panel could not clearly determine how that data was analyzed and used to identify priority initiatives.

The four district-wide curriculum directors analyzed MCAS data in their respective content areas, with the assistance of the high school Assessment Center staff, and shared it with building principals. The principal worked with subject area teachers in November 2000 to identify strengths and weaknesses using the item analysis of the MCAS provided by the state. Teachers made recommendations for improvement strategies. In the former case, there is no evidence of how extensive these analyses were and if they were utilized to inform improvement planning efforts. In the latter case, this exercise occurred after the 2000-2001 School Improvement Plan was written. The teachers reviewing the MCAS data reached general conclusions (e.g. vocabulary a weakness, open response questions weak, interpreting graphs and charts weak) but did not connect student weaknesses with specific reasons. In addition, very few strategies for improvement were generated (e.g. mathematical vocabulary and terminology review, model responses for students in all grade levels, we need test taking techniques, address areas not taught). The MCAS analysis efforts show evidence of the school's beginning attempts to use data on an ongoing basis. Since the Improvement Plan does not identify next steps, it is not known how (or if) the effort resulted in improved teaching and learning.

There is some evidence that the district curriculum coordinators have ensured that curriculum materials align with the state frameworks. The English Language Arts curriculum director decided to purchase a new text when she found that their old text, organized around genres, did not align with the State curriculum frameworks. In math, the middle schools have phased in a new program (Connected Math Program), which more closely aligns with the frameworks.

1B. Does the plan set out a course of action that is: (1) responsive to the school leader's analysis of the nature and reasons for poor student performance, and (2) is likely to lead to improved student results?

Because the analysis of data was insufficient, it resulted in action plans that may not lead to improved student results. Examination of one of the priority initiatives illustrates this point. A conclusion reached in the summary of the analysis of data specifies a priority need in the area of Reading: to provide opportunities for students to improve their reading skills, especially in the area of reading for meaning. The corresponding action plan for this initiative is: Independent Reading - implementation of the district-wide Independent Reading Policy that includes a summer reading program. The panel found no evidence of breaking down the action plan into doable steps, the results of which can be measured periodically. The connection between the need to improve students' reading for meaning skills by implementing an independent reading policy is also unclear. This initiative appears to place all responsibility for improving reading skills on students outside of school time.

In addition, the school's initiatives do not always align with the four stated district goals. District goals have separate strategies and initiatives attached to them and read as a distinct plan within the School Improvement Plan. The panel found no evidence that the Talbot initiative to move to a school-wide Title I program is tied to the district's priority goals. Nor is there evidence of analysis of program information or data leading to the initiative. A course of action is not articulated, and because it is difficult to evaluate the success of an action plan that is vague, assessment means are not clear.

1C. Was the School Improvement Plan developed through a process that will support its successful implementation?

The principal reported that he developed the School Improvement Plan using a template provided by central office that aligned it with the district improvement plan. The Superintendent confirmed that the initial part of the plan, including mission and vision statements, four priority goals, outcomes, measurements and strategies, was district-generated and is verbatim across all School Improvement Plans, with the exception of the insertion of Talbot School's demographic and assessment data into the template. The disconnect between the two parts of the plan-district-driven and school-driven-is evident (see 1B above). The development process, therefore, does not invite ownership on the part of stakeholders at the building level responsible for implementation of the improvement initiatives.

Teachers indicated that no needs assessment was conducted prior to writing the School Improvement Plan. They also stated that the faculty as a whole did not have input into the plan's content other than indirectly through School Council members. However, all teachers received a copy of the plan, and in faculty meetings they feel free to comment on improvements and changes to it. According to teachers, their principal is open to teachers' suggestions. Teacher focus groups, interviews and teacher survey results reveal priority needs not identified in the School Improvement Plan (e.g. smaller class sizes, more computers and internet access, more in-class Title I math staff, more English for ESL and TBE students, discipline issues). The fact that discipline was identified as a need but not addressed in the current School Improvement Plan is significant since five out of nine goals in both the 1998-1999 and 1999-2000 plans were focused on school climate concerns. It was notable that school climate goals have been deleted from the current plan. The Panel found no evidence that the current year's priorities were based on an assessment of the status of the prior year's goals. Parents and students, though, expressed their feelings that the school was safe and discipline not the issue that it was several years ago.

There is evidence from parent and School Council focus groups that meetings are scheduled to address district and school improvement plans. Talbot's current School Council is entirely new this academic year and thus were not involved in creating the 2000-2001 School Improvement Plan. However, according to council members, they review the document, as well as MCAS data, periodically to more clearly identify areas of need.

1D. Is the School Improvement Plan document clear and specific?

The School Improvement Plan document could more clearly articulate the connection between the goals, which were district generated, and the school's own action plans. In addition, the plan does not consistently describe outcomes and measurements for each goal, nor does it clearly specify who is responsible for each stage of the plan along a realistic timeline. Action plans and timelines for both short and long term results lack the specificity to be useful guides toward improvement.

In some cases, it is unclear what analysis was undertaken to develop the school's goals. For instance, there is no evidence of how the goal concerning the improvement of facilities was derived. The strategies articulate more specific objectives, such as developing a building plan to identify and set priorities and ensuring that building and grounds are safe. The strategies list also describes intermediate steps toward accomplishment, such as conducting an annual building audit, signing work order requests, requesting a video camera to monitor the main office corridor, and providing a female security officer. However, the plan does not always specify who will carry out the task or the timeline for completion of each step. The relationship of initiatives listed under each goal to the strategies is also unclear, especially since there are separate initiatives listed later in the plan.

Goal #4 (to ensure continuous improvement of programs and staff performance) does not articulate expected outcomes, and likewise there is no evidence of how the school will measure whether the strategies succeed in accomplishing the goal.

In summary, six out of eight initiatives identified by the school focus on teaching, learning or program concerns and the other two focus on staffing and building issues. While the areas of concentration are linked to identified weaknesses (which need to be the product of more robust analysis), the action plans are not fully articulated across grade levels and content areas. Furthermore, the Talbot's School Improvement Plan does not yet integrate district and school goals, priorities and action plans into a more cohesive overall plan for school improvement.

Key Question 2:Are The Conditions In Place For The Successful Implementation Of The Improvement Plan?

The panel found mixed evidence that the conditions are in place to successfully implement the Talbot School Improvement Plan. Existing conditions include: effective communication between faculty and building level administration; a positive school climate; expressed faculty agreement with the plan as written; and some district level resources supporting school improvement efforts, most notably the work of four content area curriculum directors. However, panel members were concerned that there is insufficient instructional leadership and supervision to ensure that improvement efforts are impacting teaching and learning in the classroom. From the evidence available to the panel, there does not appear to be adequate ongoing systemic professional development to provide teachers with the support they need and want to teach a diverse student body. Examination of the current building schedule reveals that teachers have insufficient common planning time in order to focus collaboratively on instructional concerns.

2A. Does the school have effective leadership and sound management?

The panel found evidence that the school has sound management and a foundation of resources in place that, with more focused instructional leadership, could move the school forward.

Interviews with teachers reveal that the principal communicates effectively with staff. There is dialogue at faculty meetings, and the principal is receptive to teachers' ideas. Staff, students and parents indicate that the school is safe and orderly, and observations bear that out. There was little, if any, loitering in halls, transitions between classes occurred smoothly, and panel members observed caring interactions between teachers and students both in classes and hallways. Students in one focus group communicated clearly that there was a process in place should they see a suspicious looking person in the building.

There was also evidence of high morale at the Talbot. One teacher stated, "It's fun to come to work." Others communicated their active involvement in school activities above and beyond their classroom teaching, demonstrating opportunities for teachers to assume leadership roles. Examples include representing middle school teachers on district-wide leadership curriculum committees, participation in a writing across the curriculum course and CMP (math program) summer training, initiating an after-school homework club, supporting other teachers with their technology needs, and participation in a 6:30 a.m. math focus group twice a month. In the words of one teacher, "We're moving in the right direction."

Panel members expressed concern, however, that the principal does not appear to be focused on changing instructional practice in an ongoing way. For example, he convened the faculty during an early release day to analyze MCAS scores and asked teachers to recommend strategies for improving those scores, but there was no evidence of next steps taken to ensure that changes occur at the classroom level as a result of the analysis. Likewise, teachers stated that they were encouraged to reference relevant curriculum standards in their lesson plans. However, this does not appear to be an expectation that is required across all classrooms, limiting its effectiveness. Teachers also indicated that lesson plans are collected. However, there is no evidence that feedback is given to teachers about their lesson plans or that there is any dialogue with regard to their quality.

Unfocused instructional leadership is also exemplified in the inadequately articulated process to implement a comprehensive change to standards-based teaching. While it is recognized as a priority, there was no action plan showing evidence of attention to this as a priority initiative. Similarly, the principal expressed his frustration about losing teachers who were moving toward more inclusive practices. He has plans to attend an inclusion workshop with several teachers. However, making a commitment to inclusion requires a more intensive strategy which includes communicating that commitment to stakeholders and developing a plan of sequenced steps that would help to make it a reality.

2B. Is there evidence that the school's faculty supports the planned improvement efforts?

There is evidence of faculty support for the school improvement efforts. They agreed that the improvement plan's priorities are appropriate but did not clearly articulate those priorities. For instance, when some teachers were asked what initiatives were most significant, mention was made of the move to increase the number of computers and to get Internet access. This response may indeed reflect an identified need in the school but is not a priority in the School Improvement Plan. Thus, teachers may not be familiar with the improvement priorities as specified in the plan or alternatively, may be focused on efforts that reach beyond the plan.

While the faculty supports the plan as written, interviews revealed that faculty as a whole were not involved in the assessment of identified needs. Teachers who had served on the School Council in previous years indicated that the principal had asked teachers and parents on the Council for their input. However, there was no evidence of a more formal process for gathering staff, parent or student input. Teacher interviews and survey results, in fact, identified numerous areas of need not addressed in the plan. Examples include technology, more cooperative learning, and inclusion. Staff survey results reveal that 64% of respondents identified the need for professional development in effective instructional methods for diverse learners-an area not addressed by the plan.

2C. Is the school receiving adequate guidance and support from the district leadership?

There is mixed evidence of ongoing guidance and support from district level leadership. The Superintendent, who was newly appointed in August, has simplified the district strategic plan written by the outgoing Superintendent and focused the district's efforts on four priority areas: curriculum and instruction, buildings, parent involvement, and evaluation of programs. The change in district leadership may explain the disconnect between the two parts of the School Improvement Plan although the date suggests that it was written prior to the new Superintendent's arrival. Leadership changes also include the recent hiring of a new Assistant Superintendent, who will lead the district's efforts in curriculum and instruction.

Four curriculum directors were hired one and a half years ago to provide content area leadership in English Language Arts, Math, Social Studies and Science. They meet monthly with Talbot teachers and principals in their respective disciplines. They have begun to update the curriculum and align it with the frameworks, have given teachers monthly pacing charts to help them stay on track throughout the year, chosen new instructional texts and materials, and organized training in their effective use. They also established benchmark questions to be administered to students quarterly. The Science curriculum director is developing benchmark questions for the end of each unit in Science, grades 6-8. According to curriculum directors, they have analyzed MCAS and Terra Nova results with the assistance of an assessment center at the high school. They have shared those results with the principals. While there is strong evidence of focus and collaboration among the curriculum directors, the degree to which their efforts impact what takes place in the classroom is less clear. According to the Superintendent, there is no middle school course of study although it was not clear how a course of study would fit into the larger picture of curriculum development.

There was mixed evidence, from interviews with teachers and administrators, of professional development offerings. While numerous opportunities are available to teachers, including courses, workshops and conferences within the district, at neighboring higher education institutions, and sponsored by professional organizations, there is little evidence of any ongoing systemic initiatives other than the CMP training. The district provides three days in total for staff development during the year. Most of this time is used for discrete activities, organized by administration, that last no longer than a half day. At recent staff development days, workshops were presented on topics such as the individual professional development plan, discussion of MCAS strengths and weaknesses and strategies for improvement, and changing behavior in schools. Courses and workshops are also offered in the summer. Teachers get a stipend of $18/hour for this work. However, these and other opportunities not offered within the parameters of the three staff development days are voluntary. Panel members concluded that there is insufficient sustained professional development to help teachers improve instruction.

According to teachers and principal, class sizes are large-ranging from 25-27 in core subjects to as high as 30 in other classes. Teachers express concerns that it is difficult to address individual student needs in such large classes and that there is little effort to reduce class size. Large class sizes were not universally observed. However, in some cases students had been pulled out for special education services, and number of absentees is unknown.

Finally, teachers have little, if any, common planning time. Some teachers share one period per week in common, during which time they meet with the principal (who facilitates the meeting), guidance counselor and grade level peers. At that time, according to teachers, they discuss students, interdisciplinary connections, and share strategies. However, they point out that not everyone is able to attend due to lack of universal common planning time. Curriculum directors meet with their respective subject area teachers once a month. However, the amount of common planning time limits the development of a school culture of collaboration. The Superintendent stated that he was investigating with middle school principals moving to a 6-day cycle which would allow for greater flexibility and more common planning.

Conclusion

Based on the evidence, panel members concluded that there are weaknesses in the areas of 1) data and program analysis; 2) planning of specific action steps to address needs based on that analysis; and 3) coordination of district and school plans so as to articulate a cohesive improvement effort. The panel also concluded that there is growing evidence that the conditions are in place to successfully implement the Talbot School Improvement Plan, which could further improve with stronger school improvement planning skills.

Appendix A

Team Members:

John Desses, Talbot Panel Coordinator, Massachusetts Department of Elementary and Secondary Education, Office of Accountability and Targeted Assistance.

Hazel Grenham, Director of Special Education and Pupil Personnel Services, Chelsea Public Schools, Chelsea, MA.

Paula Hutton, Principal of the James Sullivan Middle School, Lowell, MA.

Susan Rübel, Talbot Panel Chairperson, coordinator of the International Network of Principals' Centers at Harvard University, consultant for SchoolWorks, Beverly, MA.

Richard Wallace, Associate Superintendent of Schools, Everett, MA.

Appendix B

Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Please inform all school faculty and students that Review Panel members will be visiting a cross-section of classrooms during the site visit. The selection of classrooms will be determined mutually by the Panel Review Coordinator and the Principal using the staff directory information provided by the school. All faculty members are asked to be prepared to accommodate a visitor on the morning of the site visit. Panel members will make every effort to minimize the disruption of planned classroom activities.