Report of the School Panel Review of the Accelerated Learning Laboratory Worcester. MA

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low and no trend toward improved student performance is evident from MCAS data. The Accelerated Learning Laboratory was one of twelve Massachusetts middle schools meeting this criterion that were selected for panel reviews in spring, 2001. The Panel Review of the Accelerated Learning Laboratory was conducted on March 15, 2001.

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan?

The panel's responses to the two key questions that defined the scope of their review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to them. A list of panel members who participated in the Accelerated Learning Laboratory school review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Accelerated Learning Laboratory is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist, or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Accelerated Learning Laboratory Profile

The Accelerated Learning Lab enrolled 954 students in grades Pre- Kindergarten through 12 in 2000, 275 of whom were students in grades six through eight. The school identified 42 percent of its students as white, 36 percent as Hispanic, 15 percent as African-American, six percent as Asian, and one percent as Native American. Four of every five students were eligible for free or reduced-price lunch, and 44 percent spoke a first language other than English. The school identified one percent of its students as Limited English Proficient.

In 1999 (the most recent year's data available to the panel), students missed on average 10 days of school, for a daily attendance rate of 94.5 percent. During that same year, no students were reported as dropping out, while 21 students repeated a grade. In addition, the school reported that 42 students (7.7%) received one or more out-of-school suspensions, and one student (1%) was suspended in school one or more times. In 2000, four students were excluded for more than 10 days.

The Accelerated Learning Laboratory (A.L.L. School) is a Co-nect design school, with a Global Studies and Multi-Media Communications theme. The Co-nect design uses the principles of 1) shared accountability for results, 2) project-based learning - teaching for understanding, 3) comprehensive assessment for continuous improvement, 4) team-based school organization, and 5) sensible use of technology. In the fall. of 2000, the school opened its doors to a brand new facility and a doubled student enrollment. The school shifted from a totally magnet-enrolled school drawing students from across the city to one serving neighborhood families. The change in the nature of the families served represents an increase in low-income students, students from diverse ethnic groups, and students for whom English is a second language.

Staffing

The Lab's students in grades six through eight are taught by 24 full-time teachers, 12 part-time PE and Health service teachers shared with other schools; one long-term substitute, and five shared teacher leaders/curriculum facilitators for an approximate average pupil-to-teacher ratio of 7 to 1. The central staff consists of Principal Carol A. Shilinsky, a part-time assistant principal, four guidance counselors, and one librarian. The school principal is leaving at the end of this school year. Of the 31 full-time teachers, 18 (58%) have attained one or more master's degrees. The school reported that 25 of its 31 (81%) full-time teachers were certified to teach in their current position. Of the 31 full time teachers, the vast majority of them (26) have been at the school 1-2 years.

MCAS Results

Overall MCAS results showed a decline in 1999 followed by a return in 2000 to results equal to 1998. In comparing the 1998 baseline score to the 1999 and 2000 average, scores declined by four points in English Language Arts, improved by one point in Mathematics, and decreased two points in Science & Technology. In 2000, 36 percent scored in the Failing performance level in English Language Arts, 63 percent scored Failing in Mathematics, and 66 percent scored Failing in Science & Technology. Regular education students represented a lower proportion of the eighth grade class in 2000 (79%) than in 1998 (93%), and their overall MCAS participation rate declined from 1998.

Panel Responses To The Key Questions

Key Question 1: does The School Have A Sound Plan For Improving Student Performance?

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

The school implemented a partial process of data analysis in their efforts to accurately identify gaps in student performance.

The school used primarily MCAS and SAT9 performance data in their school improvement planning. As a result of their analysis, the school identified needs and began efforts to make changes in the curriculum and instructional program. For example, in response to low reading scores on the SAT9, the school has made literacy a school-wide focus. The school has adopted the Read 180 program in an effort to improve literacy skills. The low scores in mathematics reflected in the MCAS resulted in the school revising their math program from multi-age to grade level grouping. Teachers have begun to infuse learning standards from the MA Frameworks into the design of projects and the school has instituted an after school MCAS program.

While the school uses a range of assessment tools and practices in support of its mission, the information learned from those assessments does not appear to be used systematically to analyze student progress and the reasons for performance gaps. For example, it does not appear that the school has disaggregated data in efforts to understand the needs of diverse groups of students, such as those whose primary language is not English and students receiving Special Education services. There was no indication in the school's written plan or in discussions with staff that the particular needs of these student populations have been considered and are being purposefully addressed.

Factors external to the school were identified as those causing the greatest barriers to achievement at the school. The doubling of student enrollment and the inclusion of neighborhood students when the new school facility opened, coupled with high student mobility were cited as the most significant factors in the school's low scores on standardized tests. While a school analysis of student performance revealed that "old" students outperform "new" students at the A.L.L. school, one also finds that, in general, the school improved more overall on MCAS in the year 2000, the year of the school expansion, than in 1999. Conflicts between the data and the school's identification of barriers are unresolved.

There was little analysis of the impact of school-based factors on student achievement. While the school changed the instructional delivery design for mathematics, it is not evident that curricular changes have been made to address content gaps in mathematics. An analysis of math scores by gender reveals that females are performing significantly below males in math and science, yet there is no indication of a school-level awareness of that gap. In addition, teacher turnover at the A.L.L. school is significantly high, yet was not identified as an issue requiring school-level examination and action.

One of the school's two major goals is to increase parent involvement, yet the school has not made clear the gap between the current status of parent involvement and desired outcomes. The lack of an analysis of school-based factors makes it difficult to discern how the many actions noted in the plan will enable the school to better meet the needs of its student population.

B. Does the plan set out a course of action that is: (1) responsive to the school leader's analysis of the nature and reasons for poor student performance, and (2) is likely to lead to improved student results?

The plan does not provide a credible link between needs, goals, actions, intended outcomes, and mechanisms for evaluation.

The school's analysis of the nature and reasons for poor student performance was partially based on data. The primary causes for poor student performance at the A.L.L. School are cited most often as those outside the school's sphere of influence (poverty, lack of preparation at other schools, etc.). While the school did experience a significant shift in student population with the expansion and inclusion of neighborhood families, consideration of the needs of the changed population relative to the existing school programs and practices is not reflected in the school's improvement plans. For example, many staff members noted the difficulties that English language learners were experiencing in the classroom and that services were insufficient to meet their needs. The school improvement plan does not address this identified need.

The school's written plan is focused heavily on action steps and is weak in setting forth clear purposes and strategies for evaluating progress. At the same time, the school is working hard on other efforts to make changes that will lead to improvements in student outcomes. Teachers are engaged in cluster level groups to discuss teaching and learning on an ongoing basis and to monitor success in implementing school goals. Many of the improvement activities taking place at the school are not reflected in the school improvement plan. Without a clear link between goals and outcomes, it is difficult to ascertain how the school will gauge progress and to what they will credit that progress. In addition, the absence of an analysis and plan for addressing school-based factors such as teacher turnover will continue to hinder the school's ability to achieve desired outcomes.

The A.L.L. school involved key instructional staff in the improvement plan's development, through department chairpersons and team clusters, thereby enlisting ownership and support for its implementation. The involvement of teachers appears to have had positive effects on the level of faculty support of the plan. Throughout the school, teachers relayed a positive view of the school's planning efforts and a commitment to participating in its implementation. Some teachers reported using the plan at cluster meetings to discuss progress in implementation. How teams went about the process of data analysis was not clear to the panel.

It did not appear to the panel that the School Design Team, charged with overseeing the school's implementation of the Co-nect design, was centrally involved in the development of the plan. The involvement of parents and the community was limited to members of the School Site Council, where members reviewed and commented on draft plans. Parents on the Council indicated that it is primarily parents of students in the magnet program that are involved and that efforts must be made to involve neighborhood parents. In its current form, the plan is not accessible to the wider community of parents and community members. As one parent stated, "it is confusing - it is too much to read."

The current School Improvement Plan was generated, in part, from the school's prior year MCAS Acceleration Plan. No evaluation of that plan's implementation was indicated. The development of MCAS Acceleration Plans in the Worcester Public Schools was reported by administrators to be the first efforts to use data in school improvement planning. Other key school planning activities and documents were not mentioned in the development of the school improvement plan. Central to the A.L.L. school is the Co-NECT design, yet nowhere in the school improvement plan is there any reference to the implementation of the Co-NECT model as part of the school's strategies for improvement.

The apparent lack of a coherent strategy for professional development to support the actions outlined in the plans is an impediment to the successful implementation of the school's improvement plans. While the district offers many opportunities for staff to participate in professional development, it was not clear to the panel that the school has a plan in place to address the particular professional development needs of its staff in relation to the goals and requirements set forth in the school's improvement plans.

D. Is the School Improvement Plan document clear and specific?

The A.L.L. School's written School Improvement Plan is an attempt by the school to address the gaps in student performance. In its current form, however, the plan lacks focus and coherence. There is no clear prioritization of needs and actions indicated in the plan, nor are there clear strategies for evaluating the effectiveness of the actions described in the plan.

The school sets forth two broad goals (mandated by the district): 1) to improve student achievement; and 2) to increase parent involvement. The A.L.L. school's plan seeks to address these goals through a myriad of action steps in the required areas set forth by the district. There is a critical gap in the school's plan between a statement of the broad district goals and a list of broad action steps to be taken by the school. Missing is the analysis of the particular needs of A.L.L. students and faculty, the reasons underlying poor performance, and the articulation of school-specific objectives and strategies.

An incomplete analysis of data rendered the plan vague in its ability to address the needs of the school's diverse population. As noted above, a gap in math and science performance between females and males was not addressed. Nor were the particular needs of the large number of students for whom English is a second language noted. In addition, an analysis of the impact of current school practices and factors on student outcomes is missing. School factors, such as the critical teacher turnover, were not addressed at all. An analysis of current curricular and instructional practices on student outcomes is likewise unclear.

There are very few measurable goals set forth in the plan. In responding to the district form's request for "Results/Evaluations" of their action steps, the school lists data sources they intend to use without indicating desired levels of improvement. For example, in English Language Arts they indicate using writing portfolios, class discussions, and assignments to evaluate progress yet it is not clear how they will be used or how progress will be determined. Missing in the plan as well are specific timelines and processes for implementation.

Key Question 2: Are The Conditions In Place For The Successful Implementation Of The Improvement Plan(S)?

A. Does the school have effective leadership and sound management?

The school has effective leadership and management provided by the school principal and the school's Design Team. This must be understood, however, in light of the departure of the current school principal at the end of the school year.

The school principal, a founder of the Accelerated Learning Laboratory, is well respected by her school community. She has worked to support teacher leadership at the school, primarily through the empowerment of the school's Design Team in school-wide decision making and planning. The Design Team ensures that the school implements the principles of the Co-nect model and works to align the curriculum to the MA frameworks.

The school has a positive school climate, in which staff, students and parents express their support for the school and appreciation for its mission, purpose, and values. The school's new facility, along with low class sizes, provides notable structural opportunities to support learning.

A key tension exists at the school in which staff struggle to align the unique purpose of the school with the external requirements imposed by the state assessment system. The school is working to maintain the focus on project-based learning with depth over breadth while addressing the need to infuse the Massachusetts frameworks into the curriculum. A critical leadership role exists to provide a focused vision and purpose that will guide staff in moving forward in the midst of this tension. Leadership activities at present are not sufficiently integrated. The School Council, which focuses on the school improvement plan, and the School Design Team, which focuses on implementation of the school's Co-nect benchmarks, need to work in close coordination to create focus and direction for the school's improvement efforts.

A key leadership issue the school currently faces is the retirement of the school principal, a founder of the school, at the end of the school year. The capacity of the school to move forward will depend, to a large extent, on the principal who will be selected to replace the current principal. With her leadership having served as a founding and central force in that school since its inception, this change is critical for the A.L.L. school. The leadership capacity of the Design Team also will be affected by this change.

It was also not clear to the panel that the school had made any progress in involving parents and community members from their new service area in the school's decision-making processes. All staff and parents interviewed recognized the need to establish connections with the neighborhood families and community.

B. Is there evidence that the school's faculty supports the planned improvement efforts?

The school's faculty appears to have strong buy-in to the school improvement efforts. They articulate a strong sense of commitment to the purposes of Co-nect, to the school as a community, and to improve their program to better meet the needs of their students.

Although the school has experienced a significant rate of teacher turnover, the team found a strong sense of commitment across the school, from both senior and new faculty.

All faculty interviewed during the visit spoke of efforts to improve the school's curriculum to better meet the needs of their students. Teachers appear to value being involved in decision-making at the school and take their involvement seriously. There also appeared a strong sense of the personalization of student learning and in the interest in meeting student needs. Staff are committed to the principles and practices of the Co-nect design, but stated a need for more training so that new teachers can effectively implement those practices. Teachers also participate in professional networks that the school is involved in to support school improvement. Parents spoke highly of the dedication of the faculty, noting that they are accessible and are willing to respond to parents' interests and concerns.

C. Is the school receiving adequate guidance and support from the district leadership?

A range of services and supports are provided both directly and indirectly by the district to support the A.L.L. school. In some cases, however, there is a gap between the requirements and designs of centrally designed services and the needs of the A.L.L. school.

District staff, including the Superintendent, voice a high level of support for the A.L.L. school and for the continuing implementation of the Co-nect design. The district provides a range of services and supports to the A.L.L., as it does for all schools in the district.

District professional development opportunities are provided to all staff on a range of topics, many of which are offered to support district-wide goals. Federal program staff have worked to help the A.L.L. school access federal funds and programs in support of their efforts. Such examples include the school's Title I Schoolwide program and the 21st Century after school and summer programs. The district has several initiatives underway that involve or impact the A.L.L. school. Examples include a district-wide mentor program for all Worcester teachers, a homework hotline for students, and a large-scale Carnegie Turning Points initiative designed to revamp secondary schooling. District administrators stated that many of the features of the A.L.L. school design are consistent with those the district is looking to implement through the Carnegie initiative.

The district is working to support data-driven school improvement planning by designing guidelines and templates for schools to use and by providing training to school administration in how to use data in planning. The district's assessment office provides schools with a range of student assessment data to use in school improvement planning. These efforts are helping the A.L.L. school begin to use data in their planning. Their capacity to effectively analyze and use data to drive school improvement planning, however, remains quite limited at this time.

Conclusion

The Improvement Plan currently in place at the Accelerated Learning Laboratory School provides an inadequate analysis of the reasons for poor student performance. The Plan does not demonstrate a clear link between the strategies for improvement and the needs of the students, omits discussion of some important factors that affect student achievement, and fails to include a description of ways to measure the impact of various initiatives on student performance.

However, the panel members concluded that there is effective leadership capacity at the school to move the school forward. Additionally, there is a positive climate at the school, with abundant evidence of faculty buy-in to the plans included in the improvement document. In the Panel's judgment, two factors will have a critical impact on the school's ability to improve student achievement:, selection of the new school principal to replace a highly respected founding principal and stabilization of faculty turnover.

Appendix A

Team Members

Dr. Karen Angello, Panel Coordinator, Consultant to the MA Department of Elementary and Secondary Education, office of Accountability and Targeted Assistance

Carol Keirstead, Panel Chairperson, RMC Research, Portsmouth, NH

Robert Koenig, Panel Member, Curriculum, Testing and Instruction, Harwich Public Schools, Harwich, MA

Patricia Poska, Panel Member, Lead Instructional Facilitator, Lynn Public Schools, Lynn, MA

Richard Wallace, Associate Superintendent and Mathematics Director, Everett Public School District, Everett, MA

Appendix B

Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Please inform all school faculty and students that Review Panel members will be visiting a cross-section of classrooms during the site visit. The selection of classrooms will be determined mutually by the Panel Review Coordinator and the Principal using the staff directory information provided by the school. All faculty members are asked to be prepared to accommodate a visitor on the morning of the site visit. Panel members will make every effort to minimize the disruption of planned classroom activities.