Assessment/Accountability
School and District Accountability and Assistance

School Panel Review Report
Elm Park Community Elementary School
Worcester Public Schools

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low and no trend toward improved student performance is evident from MCAS data. The Elm Park Community School (Elm Park Elementary School) met this criterion at the fourth grade and was one of five elementary schools selected for panel review in spring, 2002. The panel review was conducted on March27-28, 2002.

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan(s)?

The panel's responses to the two key questions that defined the scope of its review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to it. A list of panel members who participated in the review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Elm Park Community School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist, or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Elm Park Community School Profile

One of 39 elementary schools in Worcester, the Elm Park Community School serves 467 students in Pre-K through grade 6 at two different sites. One is a comprehensive neighborhood school. The other is the Elm Park Annex, a district wide-program located about two miles from the Elm Park Community School and serving about fifty students with emotional disabilities and behavioral problems. In 2001, the school reported 39% of its students as White, 41% Hispanic, 15 % Black and 6% Asian. Students whose first language is not English (FLNE) are 42%, and 41 are % limited English proficient (LEP). Fifteen percent of the students are identified as having special needs. The school qualifies as a School-wide Title I school, with 84% of the students eligible for free or reduced-price lunch–more than triple the state average.

The Elm Park Community School has a very high mobility rate, and students move frequently within as well as outside the district. Students from ten shelters and low cost affordable housing units are served by the School.

Enrollment numbers have been stable over the past three years, as has distribution of students across reported subgroups. A notable exception is that the percentage of LEP students increased from 7% in 1999 and 2000 to 41% in 2001. The Out-of-School Suspension rate of 11.3% for 2000 is the highest among the district's elementary schools. The overall attendance rate in 2000, 94.8%, is the third lowest of ten elementary schools in the quadrant.

Staffing

The Elm Park Community School has a full-time staff of 86 that includes a principal, two assistant principals, one teacher/leader-curriculum facilitator, two guidance counselors (one part-time, and one full time), 51 teachers, 12 itinerant teachers, 27 teacher aides, and 38 full- and 7 part-time tutors. Five teachers appear not to be certified and three are on waiver. The school also has the services of a librarian, a nurse, a school psychologist and two long-term substitutes.

MCAS Results

The school failed to meet its improvement expectations in ELA and Math for Cycle 1 (1998-2000). In both 1887 and 2001, the percentage of 4th grade students scoring in the Failing/Warning performance category in ELA (63%) was equal. The percentage of 4th graders scoring in the Failing/Warning category in Math (74%) in 2001 was higher than 1998 (70%). This downward trend in student performance occurred despite a decrease in participation from 99% in 1998 to 83% in 2001. These rates are well above the district failure rates and above the state average of students scoring in Failing/Warning categories in both content areas.

The 2001 ELA data show that 7% of 4th grade students scored in the Proficient performance category, the highest percentage for the four-year test period. The percentage of students who scored Proficient in 4th grade Math increased from 2% in 2000 to 4% in 2001.

Panel Reponses To The Key Questions

KEY QUESTION 1: does The School Have A Sound Plan For Improving Student Performance?

Although its written School Improvement Plan (SIP) lacks clear goals, problem statements and specific strategies, the Elm Park Community School has a sound plan for improving student achievement that lives partially in documentation and partially in the minds of its staff and administration. Teachers at Elm Park are actively implementing a consistent set of practices that are linked to the school's broad analysis of student needs. Staff and administration recognize the need to deepen their data analysis and develop specific and measurable improvement goals for student achievement.

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

Elm Park administrators and staff have analyzed important student achievement data. The school leadership report and several interviews and teacher focus groups indicate that MCAS, Stanford Achievement Test (SAT9), and performance data from the Success For All (SFA) program (8 week assessments using the Gates-McGinities test) were all analyzed, and that some of this analysis informed the development of Elm Park's SIP. Based on an analysis of MCAS data indicating that students had difficulty in all academic areas, the Elm Park staff decided to focus on reading, writing and math. Specifically, students' open-ended responses on MCAS indicated a need to focus on writing, while student scores on the rest of the English Language Arts MCAS pointed toward the need for an increased emphasis on reading. This broad level of analysis has allowed the school to set reasonable areas of focus.

Further analysis, including disaggregation of achievement data, would allow the school to further improve its targeting of specific student needs. Based on interviews with staff and administration, it appears that to date SAT 9 and SFA data have been analyzed at the classroom level and used to inform grouping and tutoring changes in accordance with the SFA program. Further analysis of these data would more effectively inform strategic planning for the whole school. The school has a strong base of SFA data, including assessment records from other SFA schools in the district. The SFA coordinator checks with other Worcester schools for this information when new students arrive at Elm Park. To date, the school has used these data effectively to regroup students within the SFA program and to identify individual students who need additional literacy instruction. The school has not yet used them effectively to look at the reading scores for subgroups of students or to determine whether the rate of improvement in reading across the school and within subgroups is acceptable.

The principal also reports using teacher observations, lesson plans, student portfolios, and informal discussions with classroom teachers in developing the SIP. Beyond the principal's informal consideration of these factors, teachers were involved in looking at curriculum and identifying the school's strengths and weaknesses to inform the development of the SIP (for a description of how the SIP was developed, see 1E below). This programmatic review could be more systematic. It does not appear that the analysis of specific academic programs went beyond faculty and the principal applying their collective professional judgment based on informal observation. They did not systematically review key programs using a clear set of criteria. In addition, since data have not been disaggregated by specific student populations, such as English language learners, there is therefore no way to know how well the school's programs are working for each of Elm Park's various student subgroups.

The school has voluntarily engaged in the Baldrige Review with the assistance of its community partner Verizon. The focus of the review is organizational and aims to help the school understand how well it performs in seven performance categories. The results from this review are not part of the current SIP document but show the school is making efforts to measure its organizational effectiveness.

In summary, the school's analysis has identified a school-wide gap in reading, writing and math skills, and the school is looking in a general way for improvement in these areas with much more emphasis on reading and writing than math. This analysis and sense of what needs to improve appear reasonable to the panel. However, the school's analysis of data is not yet detailed enough to know how subgroups of students are performing, nor has the school sufficiently analyzed whether or not the rate of improvement in SFA scores is acceptable. Beyond the principal's informal review of programs and instruction, the school has not looked sufficiently at its academic programs and instruction and how they relate to student performance. The school has begun this type of programmatic review with its participation in the Baldrige review

B. Does the plan set out specific improvement objectives that are grounded in the school's analysis of the reasons for poor student performance?

Elm Park's student achievement goals reflect the overarching weaknesses identified in the school's data analysis and focus on student learning and parent involvement. Reading, writing and mathematics are identified as key areas of weakness, and the student achievement goals enumerated in the SIP reflect this determination. At a general level, there is a clear connection between Elm Park's data analysis and its stated improvement objectives.

Although generally sound, the improvement objectives listed in Elm Park's SIP lack specificity and measurability. Further, because no baselines of student achievement are provided, it is difficult to assess whether the specified goals are reasonable and achievable. The plan mentions several assessment measures to determine progress (under a column headed "Results and Evaluations"). For example, for the goal, "To improve student's academic achievement in English Language Arts," the SIP lists the following:

1. SFA eight-week assessments
2. Homework assignments
3. Student portfolios: benchmark improvements
4. We will decrease the gap between the percent of correct responses by students statewide and the percent of correct responses by Elm Park students
5. Student scores will increase the proficient performance level by 10% and decrease the level of failing by 10%
6. Formal and informal assessments

The SIP document does not provide the current levels of student performance based on these measures, and interviews with teachers and administrators did not give a clear or consistent indication of what an acceptable level of growth would be. For example, administration and teachers could articulate that they wanted SFA assessments to show progress but did not articulate what rate of progress would be acceptable.

When asked about Elm Park's improvement objectives, the principal clarified that goals 4 and 5 related to the 4th grade MCAS, and that for goal 4, Elm Park students would be compared to district scores not the state scores as written. These goals were not mentioned in teacher interviews or focus groups conducted by the panel. The principal enumerated three objectives for English Language Arts: 1) to increase the percentage of students reading at grade level, 2) for students to be able to complete "meaningful" sentences, and 3) for students to be able to write a paragraph in response to a prompt. These goals represent the living SIP at Elm Park, which the panel found to be sufficiently understood and supported by key stakeholders.

The learning objectives for the other academic areas covered in the plan (mathematics, history/social science, science and engineering/technology, interdisciplinary), as well as the goal of increased parental and community involvement are similarly vague with respect to having specific and measurable goals.

C. In order to accomplish each improvement objective, does the plan specify strategies which appear likely to lead to improved student results?

Elm Park's SIP does not specify strategies per se. Under the heading of "Action Steps," the SIP lists activities in which students will engage. For example, the first action step for English Language Arts is to have students "participate in SFA program that enhances vocabulary skill development, decoding, encoding, and using words with multiple meanings with an emphasis on writing meaningful sentences." Interviews and focus groups conducted by the panel review team indicate a clear understanding of what the school is doing to address identified needs. In the case of ELA, the action step that Elm Park is implementing is programmatic (SFA) and is connected to their data analysis

The action steps specified in the SIP are clear to the stakeholders interviewed by the panel review team. Success For All is obviously the primary strategy for improving student reading achievement. All teachers interviewed by the team referred to implementing SFA when asked about Elm Park's strategies for improving student achievement.

Given that they identified broad needs in their data analysis (see 1A above), focusing on implementing SFA - a comprehensive reading program - appears to be an appropriate approach to improvement. Elm Park staff consistently expressed confidence in SFA (see 2B below), and clearly have the impression that it has been successful. One teacher said, "Students have come a long way," and this statement reflects the impressions of the vast majority of teachers and parents interviewed by the panel.

On the Instructional Staff Survey, respondents rated highly the effectiveness of the school's strategies for keeping parents informed of students' academic progress and for actively involving parents. Parent focus groups revealed that the school makes commendable efforts at outreach:

• Homework is assigned daily with the expectation that parents will sign completed work.
• Nightly reading, as part of the SFA program, is implemented and monitored by parents.
• Monitoring student behavior is shared by parents and the school by sending home written reports on behavior. At Elm Park, parents ask their child or the teacher, "Did you have a green day?" This is common language for a good behavior day. The school has even begun to create contracts for helping parents with behavior at home. A student with a homeschool contract can be rewarded at school for good behavior at home.
• Several teachers and parents reported that the principal can be found about the neighborhood and helps families make sure students get to school.

While not mentioned in the written plan, continuous improvement of behavior management is a living goal at Elm Park, and there are many strategies in place. Teachers, parents and administrators all agree that the climate of the school has improved consistently over time and is now substantially better than it was several years ago. Observations around the building revealed an orderly and safe environment. The assistant principal and the behavior specialist are generally praised for their approach and the research-based strategies they have put in place.

Based on necessarily limited observations, two interviews with the ESL instructor and one interview with the principal, the panel found strategies for English language learners to be potentially inadequate and in need of further review by the school. The school has seen a 292% increase in English language learners this year (from 42 to 165), and its evening adult ESL program has grown from approximately 60 to 140 participants. There is one ESL teacher who is present in the school on part-time tutor status. She is responsible for 55 students across the grade levels and sees them for thirty minutes a day. The remaining 110 students identified a s English language learners through placement testing are not receiving services beyond SFA, which the district recognizes as an appropriate program for English language learners. There have only been two professional development activities in the last two years specifically focused on second language acquisition. One of these was a staff sharing of strategies for using SFA with English language learners (two years ago) and a one-day session on sensitivity to second language learners needs given by district personnel this year. Because SFA data have not been disaggregated, it is not clear if this level of support for English language learners is sufficient to ensure adequate progress. Given the growing percentages of English language learners in the school, the efficacy of the school's present ESL strategy is very likely to have a direct and significant impact on MCAS improvement and therefore further analysis of the present ESL strategy is warranted.

D. Are the school's written improvement planning document (s) clear and specific enough to guide their implementation of planned improvement initiatives?

Because the living school improvement plan outlines the specific skills that students need to develop, the panel reasoned that Elm Park has a sense of the issues that need to be addressed and has specified programmatic strategies to address the identified problems (e.g., SFA, MCAS after school, etc.). In the written SIP, however, these elements are not clearly articulated. Therefore, the written plan is in some regards the same and in others different than the plan that is being implemented at Elm Park. One must look at both to understand the school's approach to improvement. For example, because the SIP includes sections for History/Social Science, Science and Engineering/Technology, and "Interdisciplinary," it does not accurately represent the school's emphasis on reading, writing and math. Further, the leadership report, teacher interviews and teacher and parent focus groups indicate the importance of behavior management as part of the school's efforts, but this focus is not suggested by the SIP.

The most significant weakness in Elm Park's SIP is the lack of specific objectives (as outlined in 1B above), and the absence of measurable benchmarks. This weakness exists in both the written and living plan. Although the eight-week SFA assessments provide a solid infrastructure for establishing specific and measurable student learning goals and reviewing periodic progress, the data collected by this system is not currently being systematically analyzed for strategic planning purposes. Based on interviews with school leadership and the results from a teacher focus group, the school recognizes this is an area in need of improvement.

E. Was the School Improvement Plan developed through a process that will support its successful implementation?

The principal developed a process that provided the Elm Park staff and the parents sitting on the school site council the opportunity for input. She also clearly considered the school's previous improvement plan in developing the SIP, as indicated by a review of the previous two year's improvement plans. The school principal and most of the staff interviewed by the panel reported that everyone developed the plan together, with the majority of the work occurring at after school staff meetings (two per month). Staff reported being involved in analyzing MCAS data, identifying school strengths and challenge areas, and examining the school curriculum. All of these activities informed the development of the SIP. The school site council report being involved in developing the SIP, although respondents to the Staff survey tended to disagree with the statement that the council is an "active force behind improvement efforts."

Exactly how the written plan was communicated to various stakeholders is unclear to the panel. The ESE's Instructional Staff survey and a minority of teacher interviews indicate that a few staff members are unfamiliar with the SIP. That said, the vast majority of staff report a clear understanding of their roles and responsibilities for implementing the living plan. Support for implementing the plan, and SFA in particular, occurs both formally and informally. Formally, weekly or bi-weekly SFA "component" meetings are held in which teachers discuss individual student progress and placements, effective strategies, etc., and staff meetings are often focused on implementing SFA. Informally, the school's SFA coordinator and her assistant provide on-going support. Finally, the principal, assistant principal, the SFA coordinator and her assistant all provide "feedback sheets" to teachers after their classroom visits, and this feedback focuses specifically on SFA implementation.

KEY QUESTION 2: Are The Conditions In Place For The Successful Implementation Of The Improvement Plan(s)?

The conditions are in place for successful implementation of Elm Park's SIP. While there are some concerns about the quality of communication, the vast majority of stakeholders interviewed by the panel express support for school leadership and confidence in the direction that the school is headed. In addition, the Elm Park faculty consistently expresses the belief that the improvement strategies being implemented will improve student achievement. Finally, the school has received substantial human and financial resources from the school district that directly target both planning and implementation.

A. Does the school have effective leadership and sound management?

Staff and parents interviewed by the panel review team consistently express confidence in school leadership. Staff and parents interviewed by the panel generally praise the principal, the assistant principal, and the curriculum/technology/SFA coordinator for increasing the sense of stability and generally improving school climate.

Through interviews and focus groups, several staff reported that school leadership provides more support for teaching than has been offered in the past. Some staff specifically noted the principal's efforts to bring in different specialists to develop and strengthen curriculum knowledge and help support the implementation of specific programs. For example, the principal helped write a grant to implement the Second Step behavior program based on the recommendation of the school's behavior specialist.

In addition, results from the panel's parent and student focus groups indicate clear support for school leadership. The principal is known among the parents interviewed for her efforts to involve the community in the school and her individual attention to families. Parents interviewed also praised the improved climate at the school and the excitement among their children for the SFA program.

Some concerns were noted in the area of communication between faculty and administration. Evidence for these concerns comes from the teacher survey, the school's self-assessment in the Baldrige process and several interviews. Through a self-study as part of the Baldrige process, Elm Park has identified communication between school leadership and teachers as an area in need of attention. Results from the Staff survey show that of the 42 respondents, 29% disagreed with the statement, "Our school principal provides effective leadership to guide and support staff efforts to improve the academic performance of our students." In a focus groups and interviews, a few teachers expressed the need for clearer communication and wanted to be able to approach school leadership more easily and frequently. It is important to note that these concerns were offered along with praise for how the school's climate and academic focus have improved.

B. Is there evidence that the school's faculty supports the planned improvement efforts?

Yes, there is strong evidence that the school's faculty support the planned improvement efforts. In the Staff survey, the two statements that received the highest average level of agreement are the one having to do with having a well-defined plan (3.81 on a five point ascending scale) and the statement about the effectiveness and appropriateness of the curriculum. The Elm Park staff interviewed by the panel agrees that student's reading, writing, and mathematics skills should be the focus of school improvement efforts. They further express the belief that SFA will work if given time. One staff member remarked, "We needed a common language, and SFA got us there."

The Elm Park staff is actively engaged in implementing SFA, as well as a school-wide behavior management program. A consistent set of instructional practices and behavior management techniques were evident in the panel's classroom observations. When asked in individual interviews and focus groups about Elm Park's primary strategy to improve student achievement, teachers consistently refer to SFA. By all accounts it appears that the Elm Park staff understands what is expected of them in the classroom.

C. Is the school receiving adequate guidance and support from the district leadership?

Yes. Interviews with the Worcester Superintendent, Elm Park's Quadrant Manager, and school leaders indicate that district leadership is well informed of the needs of the school. District leadership echoes the same focus on reading, writing and math that the panel heard in its interviews with school staff and administration. The Superintendent clearly articulates a rationale for SFA being the right "antidote" for student mobility, as well as Elm Park's high concentration of second language learners. That the district pays attention to school needs is also evidenced by the fact that the district's planning process and format is reportedly being revised in response to feedback from schools.

District leadership has been directly involved in the development of the SIP. In addition to providing a strategic planning template, it also directly supports schools in creating a plan that meets the specific needs of each school's student population. Before Elm Park's SIP was approved by the Worcester School Committee, the principal received written feedback from the quadrant manager.

The district provides both human and financial resources to support improvement efforts. In addition to the feedback from the quadrant manager, the district also provides assistance with data analysis directly to Elm Park, according to interviews with district and school leadership and the school leadership report. Additional human resources are devoted to schools in the form of districtwide professional development offerings. In terms of financial resources, over the past five years the district has secured well over $500,000 to support the implementation of SFA as well as after school, summer, and adult education programs. The majority of this funding comes from federally funded Comprehensive School Reform Design grants.

Conclusion

Although its written School Improvement Plan (SIP) lacks clear goals, problem statements and specific strategies, the Elm Park Community School has a sound plan for improving student achievement that lives partially in documentation and partially in the minds of its staff and administration. Teachers at Elm Park clearly understand how the SIP translates into practice and are actively implementing a consistent set of practices. Staff and administration recognize the need to deepen their data analysis and develop more specific and measurable improvement goals with benchmarks for student achievement.

The conditions are in place for successful implementation of Elm Park's SIP. The stakeholders interviewed by the panel express support for school leadership and confidence in the direction that the school is headed. In addition, the Elm Park faculty consistently expresses the belief that the improvement strategies being implemented will improve student achievement. Finally, the school enjoys substantial human and financial resources from the school district that directly target both planning and implementation.

Appendix A
Team Members

Tom Buffett, Panel Chair, SchoolWorks, Beverly, MA
Ann-Marie Costa, Assistant Superintendent, Revere, MA
John Foscaldo, Director of Curriculum and Instruction, Foxborough, MA
Margaret Kelliher, Curriculum Management, Springfield, MA
Jack Monbouquette, Panel Coordinator, ESE, Malden, MA
Ledyard McFadden, Monitor, SchoolWorks, Beverly MA

Appendix B
Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Day 1

Day 2 : All activities take place in the school