Assessment/Accountability
School and District Accountability and Assistance

School Panel Review Report
Alexander B. Bruce School
Lawrence Public Schools

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low, and no trend toward improved student performance is evident from MCAS data. The Alexander B. Bruce School met this criterion at both the fourth and eighth grades and was one of the twelve schools selected for panel review in spring, 2002. The panel review was conducted on March 19-20, 2002.

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan(s)?

The panel's responses to the two key questions that defined the scope of its review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to it. A list of panel members who participated in the review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Alexander B. Bruce School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist, or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Alexander B. Bruce Profile

One of 20 elementary schools in Lawrence, the Alexander B. Bruce School serves 958 students in grades 1 through 8. In 2001, the school reported 88% of its students were Hispanic, 9% White, 2% Black and 1% Asian. For 86% of the student body, English is not the first language (FLNE). Thirty-two percent of the students were reported Limited English Proficient (LEP). The school has a School-wide Title I program, with 79% of the students eligible for free or reduced-price lunch as compared to 25% for the state. Ninety-five students receive special education services; 102 are enrolled in Transitional Bilingual Education, and 52 receive ESL services.

Enrollment has been over 900 students in four of the past five years with an increase of 38 students from 2001 to 2002. The student enrollment feeding into the Bruce School in 2002 is substantially different from the preceding year due to a district-wide redistricting that reassigned students to the neighborhood school. The in-school suspension rate decreased from 32.3 to 10.7 between 1998-2000, and out-of-school suspensions increased from 20.9 to 22.4. The three-year rate (1998-2000) of 20.8 was second highest in the district. The attendance rate for 1999 and 2000 was 94.0 each year as compared to the state rate of 93.8 and the district rate of 92.3 in 2000

Staffing

Alexander B. Bruce School has a full-time staff of 46, which includes the principal who was assigned to Bruce School for the 2001-2002 school year, 2 assistant principals, 2 guidance counselors, 38 teachers, and 3 teacher/leader curriculum facilitators. One bilingual teacher is on a waiver, and one English/language arts teacher was reported as not being certified. Twenty-five of the staff members have Master's Degrees and 23 hold multiple certifications.

MCAS Results

Bruce's Cycle 1 (1998-2000) Grade 4 overall performance category was "Very Low," and the school "Failed to Meet" its improvement expectations in ELA and Math. In 2001 the percentages of students in Failing/Warning in ELA was 42%, slightly less than 2000 (48%), and the number in Math for 2001 (64%) was slightly less than 2000 (66%). Thirteen percent of the students were in the Proficient/Advanced performance category in ELA for 2001 an, increase from 2000 (1%). Five percent of the students were in the Proficient/Advanced category in Math in 2001 as compared to 9% in 2000.

Bruce's Cycle 1 (1998-2000) Grade 8 overall performance category was "Critically Low," and the school "Met" its improvement expectations in ELA but "Failed to Meet" its improvement expectations in Math. In 2001 the percentages of students in the Failing/Warning performance category in ELA was 23%, a decrease from 2000 (38%), and the number in Math for 2001 (81%) is slightly less than in 2000 (88%). Twenty-two percent of the students scored in the Proficient/Advanced performance category in ELA for 2001 as compared to 21% in 2000. One percent of the students scored Proficient/Advanced in Math in 2001, as they also did in 2000.

The failure rates in both ELA and Math in grades 4 and 8 are higher at Bruce than for the Lawrence Public Schools, which is well above the state average for students scoring in the Failing/Warning category in both ELA and Math.

Important Note: Three factors are important to highlight as context for this panel review report. The Bruce School's principal, transferred to this position by the Superintendent in August of 2001, was not involved in the development of the school's 2001-2002 school improvement plan. Second, the school's plan reflects, to a large extent, an implementation plan for district goals and district-initiated reforms. Responses to the questions are not limited to school-level activity but include, by necessity, reference to the significant role of the district. Third, about half of the student/family population served by the Bruce School this year is new to the school due to the district's change to neighborhood school enrollment this year.

Panel Reponses To The Key Questions

KEY QUESTION 1: does The School Have A Sound Plan For Improving Student Performance?

Despite weaknesses in its written plan and planning process, the Bruce School is in the midst of the successful implementation of district-designed initiatives for improving student performance in literacy and is beginning to implement plans for improving student performance in math. The current school improvement plan document, developed by the former principal, was not developed through a sound school-based process of analyzing student and program data as called for by the panel review criteria. Comprised primarily of activities for implementing district improvement initiatives, the plan does not set forth a clear and focused link between the school's analysis of student needs, strategies, actions, and benchmarks and procedures for measuring progress.

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

The Bruce School conducted a limited analysis of student performance data to inform their current Comprehensive Education Plan, referencing aggregate MCAS and other standardized test data to report broad gaps in student performance. Efforts to identify underlying causes for gaps in performance were not evident in the school's planning efforts. Rather, the Bruce School plan reflects a school response to a district-wide analysis of student performance and the district's plan for addressing identified gaps.

The 2001/2002 Bruce School Comprehensive Education Plan provides limited analysis of student performance and other data, comprised of:

• aggregate performance levels of 4th and 8th grade students on the MCAS ELA, Math, and Science/Technology assessments in 1999 and 2000
• percentages of grade 1- grade 8 students scoring at or above grade-level on the Gates-MacGinitie Reading Test
• percentages of grade 3, 5, and 7 students scoring at or above grade level on the Terra Nova Mathematics assessments
• percentages of students for whom English is a second language
• numbers of student discipline referrals

No assessment of other school factors, such as curriculum, instruction, or staff qualifications, was apparent in the initial school improvement planning process. In addition there was no evidence that the school attempted to determine underlying causes for the problems identified in their analysis.

At present, the most significant data analysis for improvement planning purposes is taking place at the district level. Through the district's analysis of MCAS and other test scores, and assessment of program data, the Superintendent identified critical needs to be addressed across the system beginning in 2000/2001. His analysis of the district's most pressing needs identified "chronic underperformance" in literacy and math, high out-of-school suspension rates, a lack of measurable outcomes, and an absolute failure to meet the needs of second language learners as key areas to be addressed by all schools. The Superintendent cited a lack of clarity and coherence in the district's curricular and instructional programs as a critical underlying problem. The district's Comprehensive Education Plan sets forth goals and reform initiatives that all schools, including the Bruce School, must implement to address these systemic gaps.

Seven months into the Bruce School's improvement initiatives, there is evidence that the school (and district) is using curricular assessment data in language arts and math to identify gaps in student performance. The district is implementing a system of ongoing assessments and data analysis, including formative and summative data, to be used to guide curriculum and instructional practices. Eight-week literacy assessments and ten-week mathematics assessments are designed to provide formative data for ongoing planning purposes. There is evidence at the school that the principal and staff are increasingly using this and other data to identify and address gaps. Staff described gaps in student performance that are beginning to emerge in the eight week Success For All (SFA) literacy assessment data. The new math assessment is also described by staff as a useful tool in identifying gaps in student learning and addressing them through curricular and instructional changes. The principal described using data such as staff certifications to identify and address gaps in staffing.

Shortly before the school panel review, the school underwent an evaluation of the implementation of the Success for All (SFA) program at the Bruce School. The principal reported that this process is used to identify areas of strength and areas for improvement.

B. Does the plan set out specific improvement objectives that are grounded in the school's analysis of the reasons for poor student performance?

Because of the intentional systemic approach to school improvement being implemented in the Lawrence district, the Bruce School's plan does not stem specifically from an analysis of student performance at the Bruce School, but rather through the district-wide analysis of student and school performance. The Bruce School Plan consists of three district goals and nine district objectives with a myriad of school-level implementation activities designed to meet those goals and objectives. Staff and parents interviewed, however, expressed confidence that the district goals and objectives are aligned to the general needs of the Bruce School.

The strategic objectives in the Bruce School Comprehensive Education Plan were developed by the district and are used across the district to drive school improvement. Review of limited student performance data and interviews with school stakeholders indicate general alignment between the broad gaps in student performance identified at the Bruce School and the strategic objectives set forth in the district Comprehensive Education Plan. Plans are underway to examine how the district initiative reforms are working to meet the specific needs of Bruce School students. The school principal stated that the school's 2002/2003 Comprehensive Education Plan will be based on a school-level analysis of student performance.

The primary focus of the school improvement activity this year at the Bruce School has been the implementation of the Success for All program, adopted district-wide to address the literacy needs of students. More recently the school has been working to implement district-led math initiatives, including a new district curriculum framework and scope and sequence. The school improvement activities at the Bruce School are clearly focused on teaching and learning.

The overall goal for the Bruce School, set forth by the district and articulated by the principal, is to ensure that all students are performing at or above grade level in reading and math. With the implementation of curriculum assessments in language arts (every eight weeks) and math (every ten weeks), staff report having the necessary performance data to track student progress and to respond to student learning needs.

C. In order to accomplish each improvement objective, does the plan specify strategies which appear likely to lead to improved student results?

The improvement strategies, while district-developed, are addressing the Bruce School's priority needs to improve achievement in literacy and math. The significant focus on implementation of the Success for All program and other literacy initiatives, the recently begun math initiatives, and the development of a more coherent set of curricular and instructional practices and expectations are likely to lead to improvements in student achievement.

The Bruce School is implementing a myriad of district-initiated activities in literacy and math targeted to improve performance. These strategies include:

• implementation of the Success for All program
• adoption and implementation of a new math curriculum framework and scope and sequence
• adoption of a new grade 3-5 math program (TERC Investigations)
• before- and after-school, Saturday, and summer academic support in literacy and math
• an academic intervention program in literacy and math that provides extra help during the school day to help prepare students for MCAS literacy and math assessments
• increased staff development support through SFA facilitators, district staff developers and model classrooms

The implementation of a well-defined and consistent approach to teaching and learning appears to be contributing to improvements at the school. Staff, parents and students support the SFA program because it has helped the school to achieve a sense of purpose and clarity. They report that the program has helped the school better meet the needs of students through its grouping and regrouping practices.

There is agreement across the school that efforts must now be focused on meeting the needs of students in math. Scores in math are critically low, and staff and parents alike expressed concern that children are not receiving adequate education in math. The plans for addressing math are supported by staff, and there is confidence among staff that changes will lead to improvements in student performance.

D. Are the school's written improvement planning document (s) clear and specific enough to guide their implementation of planned improvement initiatives?

The school's written planning documents do not set forth clear statements of the problems the plan is seeking to address, nor the underlying reasons for those problems. The plan represents the school's response to carrying out the strategic objectives set forth by the district. There are a myriad of action steps; however, descriptions of the mechanisms for implementation and plans for evaluation are weak. The plan is most clear and specific in literacy, with other areas varied in their clarity and focus. Plans for improvements in math are in process and becoming clearer. Comprehensive plans for the areas of family involvement and student discipline are not clear at this time.

Many of the staff at the Bruce School saw the school's Comprehensive Education Plan for the first time in October of this year. There was little stakeholder involvement in the development of the plan, and the current principal has worked to orient the staff to the plan and the expectations inherent in it.

The primary needs to be addressed are illustrated in reports of school-wide performance on MCAS, as well as total school grade-level performance in reading and math. No disaggregation of data was evident, and efforts to determine underlying causes of poor performance were not observed. No analyses of curricular and instructional practices or other school factors were provided.

While the plan contains 50 pages of detailed action steps, the specific mechanisms for implementation, procedures for measuring progress and benchmarks are extremely limited. There was little evidence that the plan was used by staff on an ongoing basis to guide and assess progress. The principal provided a mid-year report of the plan to the team, providing evidence that a majority of the plan's activities have been carried out.

E. Was the School Improvement Plan developed through a process that will support its successful implementation?

While the school's plan was not developed through a process involving broad school stakeholder involvement, the district's systemic reforms have provided welcomed focus and direction for school staff. This district leadership, coupled with the leadership provided by the school principal in developing a school culture committed to progress, has enabled successful implementation of improvement initiatives at the Bruce School.

The school's written plan did not involve a broad range of stakeholders at the school and was developed under the leadership of the school's prior principal. The current principal has done a credible job of informing staff of the plan and of providing leadership that enables the school to successfully implement the goals of the plan and many of the proposed activities.

Beyond the written plan, the principal has worked successfully this year to develop a school culture that is committed to the goals of the plan and to improving student success at the school. Staff across the school articulated a belief in the school's direction and in the district and school leadership in place to help move the Bruce School forward. The principal stated her intent to develop broad-based involvement in the development of the school's 2002/2003 school plan.

KEY QUESTION 2: Are The Conditions In Place For The Successful Implementation Of The Improvement Plan(s)?

The clarity, focus and support provided by the district's improvement initiatives, coupled with supportive and effective building leadership provided by the principal, have created very positive conditions for ongoing improvements at the Bruce School. The school's literacy initiatives have been effectively implemented with progress evident in student assessment results. While the school has made substantial strides forward this year, it recognizes the need to improve performance in math and to address areas such as family involvement and student discipline. The school is clearly and professionally focused on student achievement and appears committed to ongoing improvement.

A. Does the school have effective leadership and sound management?

A history of turnover in leadership at both the district and school level (three principals in three years) has had enormous impact on the ability of the Bruce School to move forward in recent times, according to staff and parents. The leadership provided by the Superintendent over the past two years, coupled with the leadership brought by the current principal in the past seven months, has helped to bring a climate of purpose and focus to the school.

The leadership at the Bruce School reflects a complementary district/school leadership model in which most of the core goals and activities of the school are initiated at the district level and then implemented through leadership at the school level. The principal has the utmost confidence in the vision and mission of the district and is committed to ensuring that the Bruce School succeeds. Staff across the school expressed confidence and support for the district and building leadership. Staff supports the move toward common curricular and instructional strategies, both for the support it provides to children and for the clarity and sense of purpose and common language it is providing in their work.

The principal has provided smart leadership in helping the school to move forward this year. Recognizing the need to establish trust and confidence in her leadership, she works to successfully blend support with high expectations. She recognizes where improvements need to be made and is committed to making them in a strategically sound fashion. She acknowledges the progress that has been made at the school in literacy while recognizing the significant work that lies ahead in math.

B. Is there evidence that the school's faculty supports the planned improvement efforts?

Staff members across the school express a firm belief that the school's literacy improvement efforts are resulting in much needed improvements in student performance. While in an earlier stage of implementation, staff expressed confidence in the initiatives and plans for improving math. In other areas, such as family involvement and student discipline, there is both support for efforts being taken and concern for areas of unmet need.

Staff across the school point to the positive changes that have come about with the adoption of the Success for All program. While some faculty expressed some disappointments in the program, there is an overall feeling that the benefits outweigh the disappointments. The benefits expressed include improvements in reading, the availability of ongoing assessment data, the use of common practices throughout the school, and the professional development support provided through the program. Staff expressed an emerging concern for the effectiveness of the program for English language learners and children with special needs.

There is support for planned math improvements; however, it is too early in their implementation for staff to comment on effectiveness. Parents interviewed expressed extreme support for the literacy program and some concerns in math. Parents spoke of notable progress observed in their children's reading, while children's math work was described as low level.

Student discipline was raised as a key concern by both staff and parents. While the school has implemented both district policy directives and activities set forth in the plan, a clear and coherent strategy was not made clear. While staff expressed the opinion that the school needs to better engage families in order to succeed, they were not able to articulate how the school was poised to do so.

C. Is the school receiving adequate guidance and support from the district leadership?

The district has provided an extraordinary level of leadership, guidance and support in the Bruce School's improvement work this year. The Bruce School's improvement work is systemically linked to, and propelled by, the district plan for school improvement. The district set the improvement course for the Bruce School by creating its goals and strategic objectives and by determining several of its core elements: the literacy program including SFA and supplemental curricular offerings, the district frameworks, district assessments, academic intervention and enhancement programs.

The district provided an array of leadership as well as human, financial and material resources to support the Bruce School. First and foremost, the Superintendent assigned an experienced, capable administrator to assume the principal's role at Bruce. Other personnel supports provided include staff developers in core content areas, a district liaison assigned to the school, and personnel support for the implementation of the SFA program. The district's Assessment and Accountability Office provides the core assessments and assessment services for the Bruce School. Professional development for the Bruce School is also primarily district-developed and supported.

Conclusion

While the planning process at Bruce School lacked involvement and credible school-based analyses, and while its written plan lacks clear and focused links between needs, strategies, actions and evaluation mechanisms, the school has made great strides in implementing district literacy initiatives and is now working to focus on math improvement efforts. A combined district/school leadership model has worked effectively to bring about focus on, and commitment to, improving student achievement. The conditions appear to be in place at the Bruce School for ongoing implementation of improvement initiatives as well as for more inclusive and comprehensive planning processes.

Appendix A
Team Members

Carol Keirstead, Chairperson, Senior Research Associate, RMC Research, Portsmouth, N.H.
Karen S. Angello, Ph.D., Coordinator, Consultant to the MA Department of Elementary and Secondary Education
Shirley Alvira, Middle School Principal, Chicopee, MA
Maria Vejar-Borison, Administrative Apprentice, Lowell Public Schoools, Lowell, MA
Joyce Harrington, Assistant Superintendent, Bourne, MA
Scott Kelley, observer, Massachusetts Department of Elementary and Secondary Education Accountability and Targeted Assistance

Monitoring

Richard Warren, Monitor, Wessex Associates

Appendix B
Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Day 1

Day 2 : All activities take place in the school