Assessment/Accountability
School and District Accountability and Assistance

School Panel Review Report
Academy Middle School
Fitchburg Public Schools

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low and no trend toward improved student performance is evident from MCAS data. Academy Middle School met this criterion at the eighth grade and was one of five middle schools selected for panel review in spring, 2002. The panel review was conducted on March 27-28, 2002.

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan(s)?

The panel's responses to the two key questions that defined the scope of its review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to it. A list of panel members who participated in the review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether Academy Middle School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Academy Middle School Profile

Academy Middle School opened its doors in September 2000 to approximately 734 7th and 8th grade students. It is one of 10 public schools and one of three middle schools in Fitchburg. Due to overcrowding in the district's two existing middle schools, the city refurbished the old Fitchburg High School building and created Academy Middle, sending approximately 426 students from Memorial Middle School (now Memorial Intermediate) and 219 students from B.F. Brown.

The school board originally intended Academy to serve as a 5-8 grade school. However, several months before the scheduled opening of the school, parents registered disapproval in having 5th and 6th grade students in the same building with 7th and 8th grade students, and the city reconfigured Academy as a 7-8 middle school. The district's other two middle schools, Memorial Intermediate and B.F. Brown, serve grades 5-6 and 5-8 respectively.

In 2001, Academy's enrollment was 52% White, 28% Hispanic, 12% Asian, and 8% Black. Over half of the students are classified as low-income and eligible for free or reduced price lunch--more than twice the state average. In terms of language diversity, over one-third of Academy's students speak a first language other than English (FLNE); 55 students are enrolled in the school's Transitional Bilingual Education (TBE) Spanish bilingual program; and 43 Hmong, Portuguese, and Spanishspeaking students take ESL classes. While the high percentage of Hispanic students is reflected across the district, Academy has the highest non-white and FLNE student population of other middle schools in the district. The school also serves over two-thirds of the district's 7th and 8th grade SPED population and approximately 60% of its limited English proficient (LEP) students.

Statistical data for Academy is not established prior to the school's opening in 2000. Because Memorial Middle School contributed almost twice as many students as did B.F. Brown to Academy's population, the D.O.E. provided MCAS scores, attendance, dropout and other statistical indicators from Memorial Middle for the panel's analysis. Out-of-school suspensions between 1998- 2000 at Memorial (22.5) and B. F. Brown (14.2) were the highest of all schools in the District and well above the state average for grades 7 and 8. This trend continues at Academy in 2001-2002 where the out-of-school suspension rate has increased to 25.7. The school's attendance rate in 2001 was below both state and district averages at 89.5% as 197 students missed from 11-20 days and 194 students missed more than 20 days per year. Retention at Academy for grade 7 (.5%) and 8 (.9%) is also below both state (1.8%; 1.5%) and district (3.6%; 1.6%) averages.

Staffing

Academy's leadership team came from Memorial Middle School and includes the principal, a vice principal and a headmaster. The school's full-time staff of 82 includes 58 teachers, 48 of whom are state certified and four who are on waiver. The staff is made up of a combination of new and veteran teachers: 22 have been teaching for over 20 years and 20 for less than 10 years. The school also employs nine teacher aides, two guidance councilors, one librarian and three other professional staff. The school's 56 teachers serve 734 students for a student-teacher ratio of 12.6 to 1.

MCAS Results

MCAS data from Memorial Middle School was used as the basis for setting improvement targets and measuring performance gains. The school's 1998 baseline score of 219.7 required an overall improvement expectation of 4-6 points in Cycle 1 (1998-2000), and Academy failed to meet each of its targets in ELA, math, and science & technology. The school's overall average increase of .1 points from 1998 to 2000 resulted in a performance rating of "Very Low."

The percentage of students scoring in Proficient/Advanced in English Language Arts has shown gradual increase from 34% in 1998 to 40% in 2000 while ELA scores in warning have fluctuated. The 1998 ELA warning rate of 28% was followed by an increase in 1999 to 30% and then subsequent decreases to 26% in 2000 and 20% in 2001. In math, student warning rates show steady increases from 1998 to 2000 and then decrease somewhat in 2001. The math warning rate in 1998 of 65% was followed by increases to 66% in 1999 and 68% in 2000. In 2001, the math warning rate decreased to 60%. With the exception of the 2000 math test, Academy's warning rates in ELA and math were higher than district and state averages.

Important Note: Prior to the 2000-2001 school year, Fitchburg had two schools for students in grades 6-8, B.F. Brown Middle and Memorial Middle. In September of 2000, Academy Middle School was created for grades 7 and 8. Memorial Middle was renamed Memorial Intermediate for students in grades 5-6, and B.F. Middle added grade 5 to their enrollment to become a 5-8 school.

Panel Reponses To The Key Questions

KEY QUESTION 1: does The School Have A Sound Plan For Improving Student Performance?

Based on the evidence reviewed, the Panel found that the school does not have a sound plan for improving student performance. The Panel's review of the school improvement plan (SIP) and interviews with school administration and faculty show that the school has neither thoroughly examined student performance data nor completed a sufficient review of programs and instruction.

This weakness in the first step of effective planning hinders the school's ability to complete the rest of the planning process. Strategies have not been developed in a manner which addresses the instructional needs of various sub-groups of the population. There is no evidence of timelines and benchmarks in action planning and little evidence that specific and measurable outcomes are tracked and results utilized to inform future improvement planning.

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

The school has not analyzed appropriate data and program information to accurately identify the gaps in student performance and, therefore, has not determined why those gaps exist. Academy Middle School has available to it the results of three assessment instruments that could be used to inform instructional practices in the classroom and the development of the SIP: the Iowa test which is administered sporadically by the school, the Stanford 9 administered by the district and the MCAS. However, interviews with the principal, the school council and the teachers indicate that an analysis of these data has not been completed in a formal, school wide manner as part of developing the SIP.

The Leadership Report states that the identified critical weakness in student performance is found in the area of literacy (as determined by looking at overall MCAS results), but no further identification of student weaknesses is made through the disaggregation of data either by item analysis or by student sub-groups within the school, such as limited English proficiency.

The school has not completed a thorough analysis of its scheduling and deployment of faculty. The school is loosely organized for instruction around teams, but there is not sufficient time or flexibility in the schedule to allow for common planning times for the teachers. An outside consultant recommended that the school day be lengthened in order to provide an opportunity for teachers to meet and discuss student work and instructional practices. At this time, this change in schedule is not part of next year's planning. There are no team leaders for the grade level instructional teams and, in the staff survey, 70% of the faculty indicated that they only meet on a monthly basis to plan and coordinate.

The school has not completed a thorough analysis of its curriculum and programs. The panel learned that the district only has curriculum guides for schools in math. The Superintendent indicated that the science curriculum is in need of revision and that there is no curriculum for social studies or English Language Arts. Some of the teachers interviewed stated that their planning is guided by the state frameworks while others said that they just know what the students need.

When questioned about the reasons for poor student performance in reports and interviews, the administration, faculty and staff detailed a list of external factors such as student mobility, poverty and lack of parental involvement. At this time, there is not a school-wide recognition of the need to examine instructional practices.

B. Does the plan set out specific improvement objectives that are grounded in the school's analysis of the reasons for poor student performance?

There are no specific improvement objectives in the plan that have a specific focus on teaching and learning or that are grounded in the school's analysis of the reasons for poor student performance. In examining the SIP, the panel found that the plan has a series of non-specific goals in the area of "curriculum and instruction," such as "raise the reading achievement of struggling readers," "improvement of reading and writing achievement levels across the curriculum" and "literacy and math achievement." However, the scope of these goals is too broad, and the planning mechanisms required to achieve these goals, such as benchmarks, timelines, implementation and oversight, are unspecified.

C. In order to accomplish each improvement objective, does the plan specify strategies which appear likely to lead to improved student results?

The plan does not specify strategies which appear likely to lead to improved student results because the activities in the plan are not directly connected to identified root causes. Since a thorough analysis of student test data was not completed as part of the writing of the plan, the improvement objectives, which are listed as goals in the plan, are necessarily vague and non-specific. The plan contains a general listing of non-specific activities defined by words such as "continue," "consult," "develop," "make available" and "work towards." For example, under the goal "literacy and math achievement," the strategy listed under the heading "proposed activities" states that improvement will be accomplished by "using Title I, reading and language arts and mathematics teachers" and working "towards interdisciplinary support for literacy and math teachers in all subject areas." The panel found that this language in the plan would not assist teachers in directly impacting specific weaknesses in student performance.

D. Are the school's written improvement planning document (s) clear and specific enough to guide their implementation of planned improvement initiatives?

The school's written improvement planning documents are neither clear nor specific enough to guide implementation of planned improvement initiatives. Furthermore, since the school has not conducted a thorough analysis of student data to identify specific reasons for poor student performance, the quality of the written plan is somewhat moot.

Although Academy's SIP is articulated in writing, it does not meet the requirements of a well written improvement plan as defined by the panel review protocol. The panel found that the plan lacks a statement of the problem, an analysis of the causal factors, measurable objectives and specific strategies.

The action plan developed by the school is seriously deficient in many areas. Neither personnel nor financial resources needed for plan implementation are listed; benchmarks have not been included to check for periodic progress; specific methods of assessment of overall results have not been clearly articulated or are missing altogether.

E. Was the School Improvement Plan developed through a process that will support its successful implementation?

The SIP was not developed through a process that will support its successful implementation. The Academy Middle School does not have in place a system to involve multiple stakeholders in the process of improvement planning. The plan was written by the school's administration and reviewed by the school council. However, in its meeting with the school council the panel discovered that most of the members were not familiar with Academy's SIP, the elements of a good plan or their responsibilities regarding school improvement efforts.

Teachers only received periodic updates on the plan as it was developed. Faculty input and feedback have not been incorporated into the planning process in an organized and systematic manner. The 2001-02 SIP was only recently distributed to the faculty, and the faculty indicated, in focus groups, that they were not using or implementing the plan in their classrooms.

KEY QUESTION 2: Are The Conditions In Place For The Successful Implementation Of The Improvement Plan(s)?

The panel found that the conditions are not in place for the successful implementation of the improvement plan. A clear vision and mission statement have not been articulated by the administration. Evidence from interviews and focus groups indicate that important stakeholders in the education process, parents and teachers, were not sufficiently involved in the improvement planning process. Therefore, support for its successful implementation is uncertain. Furthermore, district-level support of the improvement planning process, as well as financial and personnel support for the school, appears insufficient to meet the improvement needs of the school.

A. Does the school have effective leadership and sound management?

The panel determined, based on evidence from the SIP, focus groups, and interviews that the school does not have effective leadership and sound management in the area of school improvement planning.

The principal of the school is an experienced educator with many years of administrative tenure in prior assignments. During the visit of the panel, she was a visible presence in the school corridors, interacting with students, facilitating panel meetings with bi-lingual parents and responding to the panel's questions. Her concern for the improvement of the school was obvious, but she has been unable, in large part, to put in place and effectively communicate a vision for improvement to her faculty, staff and parents. Results of the instructional staff survey indicate that only 53% of the staff said they were "well informed about the initiatives that are undertaken by our school leadership and staff to improve student performance." Responses in teacher focus groups corroborate this concern.

During interviews, the principal often seemed overwhelmed by the problems she saw as barriers to school improvement: overcrowded classroom, an inadequate physical plant, student mobility, student discipline, a very disorganized school opening in 2000, as well as inadequate district support. Leadership's focus on barriers rather than problem solving has inhibited the school's ability to identify issues around teaching and learning that impact student performance and are possible to influence.

Teachers and parent have not been involved in decision making relative to program improvement. Only 65% of the staff surveyed expressed confidence in the principal's ability to guide and support staff efforts to improve the academic performance of students. There are hopeful signs that some of this is changing: There is a School Climate Committee beginning to address morale issues. It is now expanding its focus to issues surrounding instruction. Also, a school retreat was recently held to provide a forum to discuss strengths and weaknesses of all aspects of the school's programs and services. Approximately twenty teachers attended this volunteer event.

B. Is there evidence that the school's faculty supports the planned improvement efforts?

There is little evidence that the school's faculty is either aware of or supports the planned improvement efforts. Evidence gathered through teacher interviews and focus groups and meetings with the school council and administration show clearly that the faculty was not involved in the planning process.

The principal stated that after the administration wrote the plan, she went over it with the school council. However, only 28% of teachers responding to the instructional staff survey see the school council as an active force behind improvement efforts at the school. The faculty received the Academy SIP only recently, and teachers stated, both individually and in focus groups, that they do not use the plan or consider it relevant to classroom instruction.

C. Is the school receiving adequate guidance and support from the district leadership?

Based on interviews with the district superintendent, his deputy and the Title I director as well as with the school leadership, the panel asserts that the level of guidance and support from the district leadership is inadequate to lead, oversee and support sound school improvement planning.

The school leadership, faculty and staff feel they were poorly served by the district when the school opened in September of 2000. As part of a redistricting effort, Academy Middle School was slated to serve grades 5-8 and occupy the building formerly used by Fitchburg High School. Administration and staff were hired or transferred from other schools, and school planning was completed with the 5-8 configuration in mind. However, following parent protests shortly before the opening of school, Academy's grade configuration was changed to 7-8. The last minute reassignment of faculty that resulted, as well as the district's lack of transition planning and support, were all cited in teacher interviews and focus groups as having negatively affected morale at the school

Additionally, student placement was reported to be disorganized and last minute. The physical plant was not ready, according to the principal, to accept students and staff into what should have been a neat, clean, orderly environment. The school feels that the district office was unresponsive to their concerns at that time.

The district does have a template for school improvement planning that identifies broad, districtlevel improvement initiatives. However, because the district has not conducted an analysis of Academy's student achievement gaps, curriculum alignment, professional development needs and classroom instruction, the panel believes that this document provides limited guidance in the school's improvement efforts. The superintendent stated that the district administration is aware that this template needs to be "tightened and strengthened" to be a more effective planning instrument. The extent to which district leaders have been involved in the development of Academy Middle School's SIP is not clear as the deputy superintendent wasn't sure if the school performed a data analysis. According to the principal, the district has not provided feedback or sufficient guidance in helping the school develop a sound and effective improvement plan.

District support of the Academy Middle School, both financially and in the form of personnel, has been limited by budgetary constraints. Interviews with the school leadership indicated that the school was chronically short of necessary textbooks and supplies and that needed building maintenance was also lacking.

Conclusion

Based on the evidence reviewed, the panel found that Academy Middle School does not have a sound plan for improving student performance. The panel's review of the school's improvement plan and interviews with school administrators and faculty show that the school has neither thoroughly examined student performance data nor completed a sufficient review of programs and, most importantly, classroom practice. This weakness in the first step of effective planning has hindered the school's ability to complete the rest of the planning process.

The panel found that the conditions are not in place for the successful implementation of the improvement plan. The school administration has not articulated a clear vision or mission statement for the school. Important stakeholders in the change process, parent and teachers, were not involved in the planning process. District level support of the improvement planning as well as financial and personnel support for the school is insufficient, at this time, to support the improvement needs of the school.

Appendix A
Team Members

Scott Kelley, ESE Coordinator, Massachusetts Department of Elementary and Secondary Education
Joe Trunk, Chairperson, SchoolWorks, Beverly, Mass.
Cindy McNally, Team Member, Assistant Superintendent, Brocton, Mass.
Eileen Kenney, Team Member, Assistant Superintendent, New Bedford, Mass.
Cynthia Palazzi, Team Member, Teacher, Agawam, Mass.

Appendix B
Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Day 1

Day 2 : All activities take place in the school