Assessment/Accountability
School and District Accountability and Assistance

School Panel Review Report
Mt. Pleasant Elementary School
New Bedford Public Schools

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether State intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low and no trend toward improved student performance is evident from MCAS data. Mt. Pleasant Elementary School met this criterion at the fourth grade and was one of four elementary schools selected for panel review in spring, 2002. The panel review was conducted on March 21-22, 2002

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan(s)?

The panel's responses to the two key questions that defined the scope of its review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to it. A list of panel members who participated in the review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Mt. Pleasant Elementary School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occurs at the next stage of the school review process.

Mt. Pleasant School Profile

One of 23 elementary schools in New Bedford, the Mt. Pleasant Elementary serves 380 students in Pre-K through grade 6. In 2001, the school reported 40% of its students were White, 46% Hispanic, 14% Black, and 1% Asian. For 23% of the student body, English is not the first language (FLNE). Twenty percent of the students are identified as having special needs. The school has a School-wide Title I program, with 89% of the students eligible for free or reduced-price lunch– more than triple the state average.

Forty-nine percent of the fourth graders tested in 2001 reported attending the school since the first grade. (Sixty-one percent of the same students reported attending Pre-K or nursery School.) Enrollment has been stable over the past three years, as has distribution of students across reported subgroups. The school reportedly has no space to accommodate in-school suspensions, and the 3-year Out-of-School Suspension rate of 9.4 from 1998-2000 is the highest among the district's elementary schools, with only two other elementary schools reporting over 5% during that time period. The overall attendance rate in 2000 of 93.5 is also slightly higher than the rate for other elementary schools in the district. Seventy-four students were absent 11-20 days and 44 students missed more than 20 days of school.

Staffing

Mt. Pleasant has a full-time staff of 42 that includes 19 teachers, a new principal, one guidance counselor, one social worker, two literacy specialists, one math specialist, and eight teacher aides. With the exception of one bilingual teacher and the guidance counselor who are on waivers, the teaching staff is certified in the area in which they are currently teaching. Thirteen of the 28 teachers also hold a master's degree. Twenty-eight teachers serve 380 students for a teacher-student ratio of approximately 14 to 1.

The staff consists of a range of teaching experience, with six veteran teachers that have taught at the school for 15-30 years, and 14 teachers with 5 or less years in the school. Among these, 12 have also been teaching for 5 years or less.

MCAS Results

The school failed to meet its improvement expectations in ELA and Math for Cycle 1 (1998-2000); and in 2001 the percentages of students scoring in the Failing/Warning performance levels in both ELA (74%) and Math (86%) were higher than they were in 1998 (55% ELA and 75% in Math). This marked downward trend in student performance included steady incremental increases in the numbers of students scoring in Failing/Warning performance levels in each of the four years in Mathematics. In ELA, a dip in the failure rates in 1999 and 2000 was followed by a large increase in 2001. These rates are well above the district failure rates, which were already well above the state averages of students scoring in the Failing/Warning levels in both content areas.

The data show virtually no movement of students from the Needs Improvement level into the Proficient or Advanced level in English Language Arts or Mathematics between 1998 and 2001. NOTE: When the previous principal resigned just before the opening of the 2001-2002 school year, the current principal began her tenure, moving from a central office curriculum coordinator position. In addition, New Bedford's superintendent began his tenure just three months ago.

Panel Reponses To Key Questions

Key Question 1: does The School Have A Sound Plan For Improving Student Performance?

The panel found that the school has made progress in its school improvement plan relative to past efforts but further work is required to develop a sound plan. The plan does generally identify student performance weaknesses in literacy and math and identifies general strategies to address these weaknesses. However, the analysis of other achievement data and data concerning program effectiveness, staffing, or current instructional practice were not sufficiently considered. While the plan's objectives and strategies are generally aligned with identified weaknesses, they lack important details such as measurable benchmarks and a systematic means for monitoring the effectiveness of both ongoing and new initiatives.

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

Important student achievement data has been analyzed. As indicated in school documents and the principal's interview, the principal and two other staff members used the Title I School-Wide Congress in August as an opportunity to review an item analysis of the 2000-2001 MCAS. (The school in its factual correction of the report noted that the assistant superintendent also attended.) Through this analysis they observed that the Mt. Pleasant students' long composition scores were significantly below the district average and that students performed poorly on all portions of the English Language Arts MCAS. Although the principal reported having helped teachers review the results of the Stanford Achievement Tests last fall, this data was not analyzed in a comprehensive way, nor did it inform the development of the SIP.

Mt. Pleasant has clearly identified student's reading and writing skills as the broad, fundamental learning needs of their students, and their SIP reflects this. The emphasis on literacy seems appropriate and the rationale for it was reinforced in interviews and focus groups with staff administration and district leaders. The focus on literacy was also supported in the results from the staff survey, and in meetings with parents and the School Site Council. The SIP also includes a goal for math, which also focuses on the reading and writing skills necessary to perform well on openended math questions.

Beyond a general understanding of a school-wide problem in reading and writing skills, the school has not clearly articulated its understanding of the reasons for poor student performance, nor has it developed clear written statements of the priority problems (see 1D below). Further, although they conducted an item analysis, their examination of performance data has not moved to the level of identifying the particular learning needs of Mt. Pleasant students For example, as opposed to determining the specific areas where students' reading skills need improvement, the school has only identified "reading" as a problem area. On the other hand, as discussed above, an item analysis did reveal specific issues with long composition in ELA.

The panel found no evidence of an analysis of programs, staffing information, current instructional practices and/or other forms of systematically collected data as part of developing the SIP. (For a description of how the SIP was developed, see 1E below.) The principal acknowledged that neither she nor others at the school have looked at these programmatic data as systematically as they would like to.

Based on a review of school documents and interviews with the Mt. Pleasant staff, the panel found no evidence of any systems in place for the formal evaluation or monitoring of the various school programs or how programs work together to form a cohesive improvement effort.

Interviews with district and school administrators and teachers, as well as open-ended responses on the Instructional Staff Survey, reflect a belief that environmental and socioeconomic factors outside of the school's control are key reasons for poor student performance. When asked what are the most significant barriers to improving student achievement (on the staff survey), staff members often pointed to the location of the school in the neighborhood of three low-income housing projects or the language skills of Mt. Pleasant students and their families. The lack of leadership provided by the previous school administrator was also cited as an explanation for poor school performance.

The team heard similar explanations in meetings with district administrators, who, despite citing the high number of second language learners at the school, stated that Mt. Pleasant has no ESL, TBE or bilingual services (see 2C below).

B. Does the plan set out specific improvement objectives that are grounded in the school's analysis of the reasons for poor student performance?

The student achievement goals reflect the overarching weaknesses identified in the school's data analysis (as reported by the school principal). Reading, writing and mathematics are identified as key areas of weakness, and the student achievement goals reflect this determination. However, because the school does not articulate a comprehensive understanding of the reasons for poor student performance (no analysis of some achievement data and no analysis of program data), this connection between the improvement objectives and the school's data analysis is difficult to establish.

Four out of the five goals set forth in the 2002 SIP reflect a focus on teaching and learning. Two of the three "School Process Goals" are centered on implementing the literacy programs Accelerated Reader and "New Bedford Reads." These reflect Mt. Pleasant's focus on teaching (the five goals are listed under 1D below). There are also two "Student Achievement Goals"–one focused on literacy and one on math.

The Stanford Achievement Test is the only concrete measure embedded in the student achievement goals listed in the school improvement plan. Because there is no baseline provided on Stanford 9 results, it is difficult to assess whether the specified goals are reasonable and achievable.

The plan does mention other assessment measures to determine progress, including running records, writing samples, benchmark books, and MCAS, but again, because there are no baselines and no connection between these assessment measures and the student achievement goals identified in the SIP, the panel could not determine whether the goals are reasonable and achievable.

The two student achievement goals lack specificity and measurability. The goals for literacy and math both state, "at least 50% of students will perform at or above grade level on the Stanford 9 and local performance measures." While this is measurable on the Stanford 9, it does not specify which "local performance measures" will be used to assess progress or how often progress will be assessed. Several school staff referred to running records as a means of on-going assessment. In its review of the written plan, and in interviews with the principal and staff, the team found no evidence that these data are being collected or analyzed in any systematic way. Furthermore, the plan does not establish a timeline for either goal.

C. In order to accomplish each improvement objective, does the plan specify strategies that appear likely to lead to improved student results?

The SIP outlines a series of "strategic actions" aimed at accomplishing each of the goals for literacy and the goal for math. However, because the root causes are not clearly articulated (see 1A and 1B above, as well as 1D below), the connection between the specific strategies selected and the results of the school's data analysis is again, only meaningful to a point. Without a complete analysis of data in hand, it is difficult to judge the efficacy of proposed strategies. But given that three-quarters of their students are failing in ELA, implementing a balanced literacy program, creating a two-hour literacy block, monthly writing prompts, and collecting and reviewing student portfolios seem likely to promote improvement in student results. All of these efforts reflect the school's increased emphasis on student writing.

The plan outlines several promising "strategic actions." The following were the most evident during the panel's visit and were discussed most frequently by the principal and school faculty:

• Implementing a balanced literacy program supported by Rigby training;
• Deploying and utilizing two literacy coaches (who focus on a combination of modeling, coteaching, resource support, working with individuals and small groups of students, and assessing student progress);
• Providing a math coach (more focused on individual student achievement and resource support than the literacy coaches, although she does provide informal professional development as well as some similar forms of support);
• Emphasizing writing school-wide (supported by the literacy coaches);
• Using monthly writing prompts (sent to and reviewed by the principal);
• Distributing computers to individual classrooms to support the implementation of Accelerated Reader (reported by the principal and teachers to have generated student excitement about reading); and,
• Implementing the Drop Everything And Read program for a minimum of fifteen minutes each day (known as "D.E.A.R." time, reported to be an effective, research-based practice). Some of these initiatives began before the current school year, and are now being fully implemented for the first time.

D. Are the school's written improvement planning document (s) clear and specific enough to guide their implementation of planned improvement initiatives?

While it shows notable improvement over the plans from the previous two years, the SIP is not written in a fashion that will sufficiently guide the planned improvement initiatives. As mentioned in 1A above, the plan is based on some data analysis but lacks a clear statement of the problem, an analysis of causal factors, and improvement objectives that are clearly measurable and time-bound.

The panel found some components of a sound plan. The SIP identifies two "Student Achievement Goals"–one focused on literacy and one on math–and three "School Process Goals." As mentioned in 1B above, four out of the five goals reflect a focus on teaching and learning. The five goals outlined in Mt. Pleasant's SIP are as follows:

For goals 1 and 3 and 2 and 4 together, and goal 5 on its own, the school's action plan enumerates strategic actions, timelines (either deadlines or how often a particular activity will happen), the people responsible, and assessment. The linkages between most of the planned solutions (the "strategic actions") and the improvement objectives are reasonably clear, although it is not always obvious if the actions are designed to directly support student achievement or indirectly support it through the realization of goals 3 or 4 (school process goals).

There appears to be confusion regarding how the effectiveness of implementation will be determined. Under a header of "Assessment," the school action plan offers the following question: "How, when will we measure the impact of our action?" For each strategic action, however, the action plan describes how implementation will be monitored, not the effectiveness of the action. For example, for assessing the effectiveness of the D.E.A.R. program (mentioned in 1C above), the plan indicates that teachers will post the time that D.E.A.R. will occur and the principal will check. This is an effective strategy for monitoring implementation but not for assessing the effectiveness of the sustained silent reading program.

E. Was the School Improvement Plan developed through a process that will support its successful implementation?

The principal developed a process that provided all staff (and some parents) the opportunity for input. He considered the school's previous improvement plan in terms of the relevance of proposed actions, the sheer number of initiatives, and a realistic assessment of staff capacity. In the words of one school site council member, "This year's plan is more specific and more do-able...We've had more input than before."

According to the principal, the process for developing the SIP was one of paring down and refining. After making some cuts in the 2000-2001 plan, the principal shared the revised version with two staff members who had traveled with her to the Title I School-Wide Conference; and the three of them scaled back the revised plan further. The plan was then shared with the entire staff for comments and suggestions, and further revisions were made. The latest version (reviewed by the panel) is also currently being reviewed by central office (see 2C below), and the principal describes the process for its development as "on-going." All of the staff interviewed by the panel reported having had the opportunity to provide input.

Although the plan's development was described as a work in progress, it is unclear how or on what basis future modifications will be made. As described in 1A above, while there are many data streams that could be utilized for assessing the implementation and effectiveness of the strategic actions outlined in the plan (e.g., running records, results from the Accelerated Reader, etc.), the panel found no evidence of a systematic monitoring effort in place or evidence of plans to institute any.

Key Question 2: Are The Conditions In Place For The Successful Implementation Of The Improvement Plan(s)?

Yes. At present many of the conditions appear to be in place to enable the school to implement a sound plan for student improvement. On all accounts the school has made significant positive changes this school year. Central office, staff and parents all express high levels of support for the new school principal, and the panel was impressed with her strong leadership skills. In interviews and focus groups, staff expresses confidence in the principal and the direction she is moving the school.

While central office has provided substantial resources to support improvement planning, the new acting superintendent and the assistant superintendent did not appear to be well informed of the specific needs of the school or of the reasons for student's critically low performance. In addition, it is unclear whether the school's current level of resources could be sustained with imminent budget cuts referred to by the superintendent. In the absence of substantial and sustained support from the district in the form of human and fiscal resources, it appears unlikely that the school can maintain and build on the initiatives already underway or plan for new initiatives as needed.

A. Does the school have effective leadership and sound management?

Yes. The school appears to have effective leadership. In a very short time, the new principal has energized and galvanized the school staff. Several staff members and parents interviewed by the panel commented on how much better the school is this year than it was last year. The principal's previous position at central office as a curriculum coordinator provided her with some familiarity of student work through monthly writing prompts sent to the central office. She also appears to have gained the trust of the Mt. Pleasant staff through her solid communication, curricular expertise, "high visibility," "strong organizational skills," and a judicious and "fair" disposition. Staff rated the principal's effectiveness in guiding and supporting "staff efforts to improve the academic performance of our students" very highly on the staff survey (the average staff response was 4.83 on a five-point ascending scale). Panel interviews and focus groups with staff members corroborated this result.

The school staff has confidence that the principal is leading the school in a positive direction. By all accounts the principal has vastly improved the school climate through her accessibility, as well as her consistency with regard to student discipline. Parents participating in a focus group reported that, compared to last year there is better discipline, parent-administrator relations, and morale among the teachers. Some teachers interviewed referred to a sense of division in the school under the previous principal. Many of the staff interviewed by the panel indicated that things are very different now. As one teacher reports, "She [the principal] has good leadership skills. She gets back to you. She works on getting grant money and support from the district." Another teacher reports that the school "Always had good intentions and desire to have them [students] succeed, but they never had the guidance and leadership. Now there is not as much conflict."

B. Is there evidence that the school's faculty supports the planned improvement efforts?

Yes. Based on individual teacher interviews and staff focus groups, school staff emphatically supports the school-wide focus on literacy. Although they were not all actively involved in identifying the needs to be addressed through the planning effort, no staff members interviewed by the panel indicate dissent–all express support for targeting student's language skills through an increased focus on literacy.

Most staff interviewed by the panel express support for the improvement strategies being implemented, as well as the belief that the school is on the right track. As one teacher puts it, "I feel hopeful. I see the growth that has occurred in the little ones." When staff were asked whether the school "has a well-defined plan for reaching student performance goals" on the staff survey, the average response was 4.46 on a five-point ascending scale. A meeting with the School Site Council further corroborated this result.

C. Is the school receiving adequate receiving adequate guidance and support from the district leadership?

While the principal reportedly receives extensive support from central administration, the panel found mixed evidence of this. District administrators reported that although the Mt. Pleasant School serves a disproportionate number of second language learners, the school does not have a second language program; in fact, it offers no language services because parents are not interested in them. Yet, a small sample of parents interviewed by the panel expressed a desire to have more support for English language learners. There is, however, evidence that central administration is carefully considering the district's support for English language learners. On the second day of the panel review, the assistant superintendent delivered an "Evaluation of the Transitional Bilingual Education Program" completed by Brown University last fall; however, the panel could not carefully analyze its contents due to time constraints.

Although the district provides substantial human and financial support to Mt. Pleasant in the form of math and literacy coaches, the panel did not gather evidence of direct involvement by the district in the school's improvement planning process. After the visit, the district reviewed this report for factual inaccuracies and replied that the assistant superintendent participated in the Title I School Wide Congress last summer and helped develop the school improvement plan. The district provides professional development for administrators, including consulting support directly focused on strategic planning, but to date it has not provided substantive feedback on the planning process or the plan itself. (It is important to note that the plan is currently under review for approval by the central office, and the panel cannot predict what feedback might be forthcoming.)

The district also provides the principal an experienced mentor principal, on-site consulting services focused on strategic planning, the funding to support the curriculum coaches (two literacy and one math), and TestWiz, a software package designed to aid data analysis. With a financial deficit looming, several staff members express the concern that current resources will be cut, and improvement efforts undermined as a result.

Conclusion

While there are several improvement initiatives underway, the current SIP lacks clear and reasonable goals, measurable benchmarks, and definitive timelines. The current plan is not based on a thorough analysis of all relevant data and does not include a systematic means for monitoring its effectiveness. The plan is currently under review for approval by the central office.

Despite an improvement plan that lacks crucial elements, several of the conditions for successful improvement appear to be in place. By all accounts, the school has made significant positive changes this school year. Staff and parents express high levels of support for the new school principal, and the panel was impressed with her strong leadership skills and her grasp of the academic challenges ahead. Although the principal is only five months into her tenure, the staff expresses strong confidence in her and in the direction she is moving the school. While central office provides substantial resources to support improvement planning, district leaders could be better informed of the specific needs of the school. It is not clear whether the school's current level of resources can be sustained.

Appendix A
Panel Members

Tom Buffett, Panel Chair, SchoolWorks, Beverly, MA
Denise Delorey, Program Coordinator, ESE, Malden, MA
Terri Caffelle, Curriculum Specialist, Shrewsbury, MA
Kate Fenton, Director of Professional Development, Springfield, MA
Marcia Pertuz, 3rd Grade Teacher, AMIGOS School, Cambridge, MA

Appendix B
Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Day 1

Day 2 : All activities take place in the school