Assessment/Accountability
School and District Accountability and Assistance

School Panel Review Report
Bartlett Middle School
Lowell Public Schools

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low, and no trend toward improved student performance is evident from MCAS data. The Bartlett Middle School met this criterion at the eighth grade and was one of five middle schools selected for panel review in spring, 2002. The panel review was conducted on March 19-20, 2002.

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan(s)?

The panel's responses to the two key questions that defined the scope of its review are included in this report. These findings and conclusions are the product of the panelists' analysis, discussion, and observation, based on the evidence available to it. A list of panel members who participated in the review is provided in Appendix A. An illustrative schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining if the Bartlett Middle School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Bartlett Middle School Profile

One of seven middle schools in Lowell, the Bartlett Middle School serves 608 students in grades 5-8. In 2001 the school reported 24% of its students were White, 52% were Asian, 20% were Hispanic and 4% were Black. Thirteen percent of the student body was enrolled in the Cambodian Transitional Bilingual Education program. English is not the first language (FLNE) for 63% of the students, and 15% have limited English proficiency (LEP). Fifteen percent of the students have learning disabilities, developmental delays or emotional/behavioral disorders. The Bartlett Middle School qualifies as a school-wide Title I school with over 82% of students receiving free or reduced price lunch. Over the last three years the school's Title I budget has exceeded $500,000. Over $400,000 is spent on salaries for instructional and support staff, and the remainder is expended on supplies and other services.

The enrolment has varied from 641 students in 1998 to 574 students in 1999 and 2000 to 608 students in 2001. The out-of-school suspension rate of 27.7% for 2000 was the second highest of the seven middle schools in Lowell, and the exclusion rate of 12.2% was the highest of all schools in Lowell and double the exclusion rate of the high school. The attendance rate of 93.6% for 2000 was the third lowest of the seven middle schools.

Staffing

The Bartlett Middle School has a full time staff of 52 that includes a principal and an assistant principal, a behavior modification specialist, a guidance counselor, a teacher leader/curriculum facilitator, a long term substitute and 47 teachers. Twenty-eight teachers have earned one or more master's degrees. Four teachers have waivers and one is not certified. In addition, there are full-time and part-time aides.

MCAS Results

The school failed to meet its 'overall' improvement expectations for cycle 1 (1998-2000). In 2001, the percentages of students scoring in the Failing/Warning performance level in both ELA (31%) and Math (77%) were higher than they were in 1999 (ELA 29%, Math 75%). The percentage of students scoring in the Failing/Warning performance level (31%) in the 2001 ELA test was also higher than the Failing/Warning percentage in the 2000 ELA test at 24%. The percentage of students (77%) in the Failing/Warning performance level on the 2001 Math test was lower than the Failing/Warning percentage (83%) on the 1998 Math test but higher for both 1999 (75%) and 2000 (72%). These numbers indicate mixed results over the four-year period, whereas the Failing/Warning percentages in ELA and Math for all 8th grade Lowell students show a clear trend of improvement.

Panel Reponses To The Key Questions

KEY QUESTION 1: does The School Have A Sound Plan For Improving Student Performance?

Based on the evidence reviewed, the Panel found that the school does not have a sound plan for improving student performance. The Bartlett Middle School Improvement Plan (SIP) has been written using a district issued "Unified School Improvement Plan" template. In its SIP, the Bartlett has met the requirements of the approved district template; however, the template does not require sufficient information. Based on the Panel's review of the school improvement plan and interviews with school administration and faculty, the school has examined student performance data but has not completed a sufficient review of programs and, most importantly, classroom practice. This weakness in the first step of effective planning hinders the school's ability to complete the rest of the planning process: Strategies have not been developed in a manner which is likely to address the instructional needs of various subgroups of the population. There is no evidence of timelines and benchmarks in the action plans and little evidence that specific measurable outcomes are tracked to inform future improvement planning.

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

The school staff has participated in various activities, including Saturday data workshops, to analyze MCAS and Terra Nova test results. The faculty has completed item analyses of the available data sets and identified gaps in student performance. However, the school has only recently begun to make the connection between gaps in student knowledge and necessary improvements in instructional practices. Most importantly, the school has not conducted a sufficient, objective review of classroom practice as part of its school improvement planning process. Most of what the school leadership knows about current instructional practices comes from teachers' self-assessments. These can be helpful but are not a substitute for a system of accountability based on classroom observations and evaluations by the school's administration. The leadership report indicates that teachers have asked for this feedback. The principal recognizes it as a responsibility that she has not been able to fulfill. Teachers indicated the same need on the Panel Review teacher survey, rating "effective instructional methods for diverse learners" as their most significant professional development need. District assistance through additional resource staff promises to alleviate this problem.

In the opinion of the panel, the school needs to inventory, evaluate and prioritize the various programs and initiatives presently underway. The effectiveness of previous school improvement planning efforts is not cited in this year's action plan. The need to set instructional priorities becomes clear when one considers the following examples of shifting focus in the school improvement plan: References to the Links program and writing prompts as well as the Wilson Reading Program come and go in the action plans. Literacy has reportedly been the main focus of the school since l998, but a 5th and 6th grade integrated model to improve literacy skills mentioned in the 2000-01 Action Plan is not found in this year's plan. While mathematics has now joined literacy as a focus, a major five-year technology initiative was begun in 2000-01 requiring a great investment of teacher time in technology training. While the initiative is commendable, it appears outside of the school's key academic needs.

B. Does the plan set out specific improvement objectives that are grounded in the school's analysis of the reasons for poor student performance?

The school improvement plan's objectives are neither specific nor grounded in the school's analysis of the reasons for poor student performance. Based on document review and interview information, the school appears to be at an important juncture between understanding student performance data and exploring to what extent and in what ways the school's curricular and instructional practices are responsible for present levels of student performance.

The goals in literacy and mathematics state that: "all students will learn high level skills, concepts and habits needed to succeed." The expected improvement in literacy is a minimum of one year and baseline data is still being collected for mathematics. There is no reference to problem areas identified through data analysis of the MCAS, particularly item analysis.

The Panel's determination that the plan's objectives lack specificity make moot questions as to whether or not the objectives are reasonable, achievable and measurable.

While the present plan does not do so, there are extremely hopeful signs of focusing efforts on teaching and learning. These include the school's recent success in securing a CSRD grant to fund the Turning Points initiative, professional development in utilizing Looking At Student Work protocols and a math initiative guided by professors from UMASS-Lowell and Framingham State College. The school's leadership recognizes the need for narrowing efforts to address instructional practice, and while their efforts presently lack focus, the faculty appears motivated to improve.

C. In order to accomplish each improvement objective, does the plan specify strategies which appear likely to lead to improved student results?

Suggestions, resources and activities have been referenced in the action plans. Where strategies are selected, they are not directly connected to the identified root causes. Strategies that are selected need more detail for effective implementation. Not all of the strategies are research based, nor do they have a demonstrated level of effectiveness

With the above concerns in mind, the school is engaged in a number of improvement initiatives. Through a teacher led and administration supported effort, the school has adopted block scheduling for its 7th and 8th grades to provide teachers larger amounts of time for instruction. Common planning time on a daily basis is also available at all grade levels and has been for ten years. Other middle school practices utilized are cooperative learning, advisory periods, peer mediation and tutoring.

The 5th and 6th grades are self-contained and teachers at these grade levels are considered generalists. The district and school are beginning to address their professional development needs so as to provide the same level of support to teachers at all grade levels.

The school has done a commendable job in providing support services to its students to insure that the individual needs of students and families are appropriately addressed. Some examples are SPED support for students with disabilities and a transitional bilingual education program for the Cambodian bilingual population. There is also a bilingual (Khmer) parent resource staff member who has helped to involve more Cambodian parents in the life of the school and as partners in their children's education.

Professional development is encouraged and supported at the Bartlett school. Faculty members are constantly involved in professional activities both in and outside of school, and new instructional ideas are readily shared with interested faculty. There is a collegial network that has evolved over time and improved morale, risk taking and experimentation with new strategies in the classroom.

D. Are the school's written improvement planning document (s) clear and specific enough to guide their implementation of planned improvement initiatives?

The analysis of causal factors noted in the SIP largely focuses on external issues such as demographics, prior schooling issues, parent involvement, language diversity and poverty levels. The analysis does not sufficiently address instructional issues. This limits the ability of the writers of the document to produce objectives and specific instructional strategies which will improve student performance. There are no benchmarks established in the action plans. Though there are general methods of evaluation, it is not clear what will be done with evaluation results. Action plans need to be expanded to include specific steps, realistic timelines for completion and identification of persons responsible for implementation.

The team found the SIP difficult to follow and not particularly user-friendly. This is a concern for groups such as community members and parents wishing to follow a particular strand. For example, to follow one strand, you must read the action plan, then the review section, then the overview of the next year's plan and then the next year's action plan- constantly switching back and forth for necessary information. The complicated form has resulted in some inconsistencies in the plan. Very often it seems that the narrative review of a previous year's plan includes a summary of the past years activities that may or may not be mentioned in the action plan itself but are critical to program improvement. For example, in the l999-00 plan, a new science text series was adopted, but professional development for the series was not part of the action plan. However, professional development for teachers in the use of that very series does appear in that year's review of the plan. A cleaner, clearer plan template would add a great deal of transparency to the SIP process.

E. Was the School Improvement Plan developed through a process that will support its successful implementation?

Although its analysis of instructional practice, selection of strategies, and action planning are not sufficient, the school has made good efforts to produce a plan through an inclusive process. The faculty was involved in the analysis of MCAS test results identifying student strengths and weaknesses and was invited to submit suggestions for activities to be part of the plan. Staff also responded to district surveys. More informally, the faculty was encouraged to provide input to the plan during team meetings and discussions. The school site council also gave input, and parents responded to district surveys. The completed plan has been communicated to all stakeholders.

The faculty did not actually write the plan, and the panel was unclear as to who authored the SIP. When the faculty returned to school in August, they spent an orientation day reading the completed plan and commenting on whether it actually reflected the reality of the school. The panel believes that this step of the process needs to be revisited because several teachers expressed confusion as to the role and purpose of the SIP as it relates to instruction. This may indicate that some percentage of faculty members don't understand their role in and responsibility for implementing action steps in the SIP.

Prior to this school year, the format of the SIP required a narrative review of the previous year's plan under the headings that appear in the action plan: Curriculum, Professional Development, School Climate and Parent Involvement. This was followed by a narrative overview of that year's plan with the action plan immediately afterwards. Although previous improvement plans were discussed, indications of their success or lack of it were not always documented. There was no mention of a regular review of progress in plan implementation, and therefore presumably no process for making adjustments when needed.

KEY QUESTION 2: Are The Conditions In Place For The Successful Implementation Of The Improvement Plan(s)?

Although there are significant concerns about the template used to develop the current SIP, the conditions are in place for the successful implementation of improvement initiatives. The school leadership has the support of the vast majority of stakeholders in the school. They, in turn, have the confidence and motivation to affect improved student achievement.

A. Does the school have effective leadership and sound management?

The principal is energetic, caring and focused on improving the education of her students. The faculty, based on the interviews and staff surveys, recognizes the principal as the school's instructional leader and expresses confidence in her ability to lead improvement efforts.

The principal has created many opportunities for the dissemination of information and for the various stakeholders in the school to be involved in the improvement dialogue and decision making process. For example, each grade level has a team leader who chairs common planning times for their teams. Team leaders then meet once a week, after school, with the principal to discuss administrative and educational issues. There is also an administration team which includes nonteaching staff including the nurse, social worker, parent liaison and a community member. Additionally, there are math and literacy teams, and an instructional technology team is being formed.

In focus groups, school site council members and parents also expressed their confidence in the principal's leadership and related how they are informed about and involved in the school's improvement efforts. This openness has no doubt contributed to the high morale and positive atmosphere that are seen and felt in the school. Teachers feel supported and empowered to investigate new and promising educational practices. To build on this existing base of support, it is the judgment of the panel that, although existing individual and group initiatives are beneficial, the principal needs to more narrowly focus improvement and professional development efforts on instructional practices in targeted curriculum areas.

B. Is there evidence that the school's faculty supports the planned improvement efforts?

The evidence gathered by the panel indicates that the school's faculty supports improvement efforts. Instructional staff survey results show that they feel that the school has a well-defined plan for reaching student performance goals, that the curriculum is effective and appropriate and that they are well-informed about student improvement initiatives. While the panel disagrees with these specific claims when considering the written SIP, it did find teachers open to improvement and engaged in meaningful improvement activities.

The faculty has been involved in data analysis of the MCAS results for several years, and they are well aware of the needs of the students relative to the state's performance standards. They have been working hard to implement current improvement efforts in English Language Arts since l998-99. All staff received training in the Links program, which provides skills in the use of graphic organizers. The improvement in the students' writing skills has been attributed to that effort. The faculty feel that they can be equally effective in their new focus on math because "it worked with ELA."

Due to its complex structure, the school improvement plan template is not serving as a good vehicle to express measurable benchmarks for the improvement efforts underway at the school. The district and school leadership need to examine this disconnect between the tremendous amount of planning and analysis work that teachers do and the written school improvement plan so that the written plan accurately represents current initiatives and provides a better instrument for evaluating the effectiveness of initiatives over time.

C. Is the school receiving adequate receiving adequate guidance and support from the district leadership?

The principal of the Bartlett School has indicated that the level of guidance and support from the district office has increased with the arrival of the new superintendent, a short year and a half ago. In support of Lowell' s schools, the Superintendent has reorganized the central office, selected a new deputy superintendent and expanded that position's responsibilities from curriculum and instruction to curriculum, instruction and assessment. In addition, she has created the new positions of K-12 coordinators for English language arts, mathematics, and research, testing and assessment. The Superintendent praises the leadership at Bartlett School and has expressed faith in its ability to affect needed changes.

During the interview with the Superintendent, she said that both her office and a sub-committee of the school committee review the improvement plans of all schools. She feels that, as presently designed, the plan is written through a "bottom up process." To improve the planning process, she would like to move to a format of S.M.A.R.T. goals (Sustained, Measurable, Attainable, Relevant, Time-bound) and to provide action planning templates with timelines and measures of effectiveness.

In addition to the creation of new district positions, the Superintendent will assist the Bartlett School by refocusing some district resources to support the hiring of an additional mathematics teacher. Finally, the Superintendent hopes to develop system-wide initiatives in English Language Arts and mathematics so as to create accountability for student progress and to measure effectiveness. She felt that in the past the district office has not been organized to sustain such initiatives, leaving many schools to their own devices.

Conclusion

The panel finds that the Bartlett Middle School does not have a sound plan for improving student performance. In its SIP, the Bartlett School has met requirements of the approved district planning template; however, the template does not require sufficient information. In particular, the analysis of instructional practice is insufficient, and therefore strategies have not been developed in a manner which addresses the instructional needs of various sub-groups of the student population. There is no evidence of timelines and benchmarks in the action plan. There is little evidence that specific measurable outcomes are collected and utilized to inform future planning.

Although the above issues exist with the current written SIP, the panel believes that conditions are in place for the successful implementation of improvement initiatives once existing initiatives are better organized within a clearly written SIP. The school leadership has the support of the vast majority of stakeholders in the school. They, in turn, have the confidence and motivation to effect improved student achievement. There is evidence that the school is presently engaged in some meaningful improvement efforts. The district is also providing good leadership and support.

Appendix A
Team Members

Jack Monbouquette, Coordinator, Consultant, Massachusetts Department of Elementary and Secondary Education
Joe Trunk, Chairperson, Consultant, SchoolWorks
Hazel Grenham, Director of Pupil Personnel Services, Chelsea, Ma.
Cheryl Despirt, M.S. Principal, Springfield, Ma.
William McGowen, Administrative Apprentice, Lowell, Ma.

Appendix B
Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Day 1

Day 2 : All activities take place in the school