Assessment/Accountability
School and District Accountability and Assistance

School Panel Review Report
The Benjamin Banneker Charter School
Cambridge, Massachusetts

Introduction

The purpose of the School Panel Review Process is to assist the Commissioner of Education in determining whether state intervention is needed to guide improvement efforts in schools where students' MCAS performance is critically low and no trend toward improved student performance is evident from MCAS data. The Benjamin Banneker School met this criterion at the fourth grade and was one of 12 schools selected for panel review in spring, 2002. The panel review was conducted on April 2-3, 2002.

The review panel's charge was to analyze data and written information on the school's performance and improvement efforts, visit the school, and meet with school and district officials in order to advise the Commissioner on the answers to the following two key questions:

1. Does the school have a sound plan for improving student performance?
2. Are the conditions in place for the successful implementation of the school's improvement plan(s)?

The panel's responses to the two key questions that defined the scope of its review are included in this report. These findings and conclusions are the product of the panel's analysis, discussion, and observation, based on the evidence available to it. A list of panel members who participated in the review is provided in Appendix A. A detailed schedule of the panel's activities is provided in Appendix B.

The panel's findings and conclusions on the two key questions will be forwarded to the Commissioner of Education for consideration, together with school performance data, in determining whether the Benjamin Banneker Charter School is deemed under-performing. The panel was not asked to formulate a sound plan for school improvement where such a plan does not presently exist, or to recommend a course of action to create the conditions for successful implementation of sound improvement strategies where such conditions at present do not appear to exist. Diagnostic and/or prescriptive intervention, where needed to assist an under-performing school, occur at the next stage of the school review process.

Benjamin Banneker Charter School Profile

Located in Cambridge, Massachusetts, the Benjamin Banneker Charter School (BBCS) enrolled 328 students in 2001 within a Grade K-8 configuration. The grade configuration of the BBCS has expanded with the addition of one grade level each year since 1996 when the school served students in grades K-5 until 1999 when the school reached its current K-8 configuration. Enrollment figures within this grade level span have increased from 290 students in 1999 to the current figure of 328 students. In the 2001 MCAS Student Questionnaire, 56 percent of fourth graders reported attending the school since the first grade; 71% of eighth graders tested reported attending the school since grade 5 or earlier.

The BBCS student body is 93% African American, 5% Hispanic and 1% Native American. BBCS has a School-Wide Title I program with 57% of the student body being eligible to receive free and reduced price lunch. The school reported that 19% of its students speak a first language other than English, which represents an increase from 11% in 1998. Approximately 11% of students receive Special Education services.

In 2000 (the most recent attendance data available to the panel), the Benjamin Banneker Charter School's daily attendance rate was 93.7 percent. Students missed an average of 11 to 12 days per school year, a rate that has remained steady over the last three years. The retention rate at BBCS has decreased from 6.2% in 1999 to 1.3% in 2000 when four students were retained in grade in contrast to eighteen retentions the previous year. The rates for out-of-school suspension have fluctuated from 18.8% in 1998, then decreasing to 10.7% in 1999, and climbing back to 15.6% in 2000. Fortynine students received out-of-school suspensions in 2000. In-school suspensions increased significantly from 1999 when three students received in-school-suspensions (3%) compared to 11.4% in 2000 when 36 students were suspended in school.

Staffing

Benjamin Banneker Charter School students are taught by 37 full-time teachers as well as 10 teacher aides. The central staff consists of an interim executive director, an assistant principal, curriculum director and director of special education. Approximately one-half of the faculty (17) hold master's degrees. The school reported that 18 of the full-time teachers were not certified to teach in their current position and that approximately two-thirds of the faculty have been teaching two years or less in the school.

MCAS Results

The Benjamin Banneker Charter School's overall performance Category for Cycle I was "Very Low" and the school failed to meet its improvement expectations for 1998-2000 at the grade 4 level. Overall grade 4 MCAS results revealed a decrease of 4.3 points in the average scaled score from a 226 baseline in 1998 to a 1999/2000 average of 221.7 points. Approximately one-third of the fourth graders scored in the Warning performance level in English language Arts from 1998-2000. In 2001, the number of fourth graders in the Warning performance level increased to 55%, a rate that is five times the state percentage in ELA. Grade 4 MCAS results in Mathematics fluctuated from 1998-2001. The 2001 results saw an increase in the number of fourth graders in the Warning performance level to 66% from the 1998 baseline of 47%. The percentage of students scoring in the Proficient/Advanced performance levels declined in both content areas from 1998 at the grade 4 level. Participation rates of 4th graders on the MCAS tests declined slightly from 100% in 1998 to 97% in 2001.

Grade 8 MCAS results from 2000 to 2001 demonstrated significant improvement in both English Language Arts and Mathematics. The number of students in the Warning performance level in English Language Arts decreased from 16% to 0%. State percentages for students scoring in the Warning performance level in 2000 and 2001 were 11% and 8% respectively. BBCS students scoring in the Proficient/Advanced performance levels increased from 48% to 55%. Results in Mathematics during this same period of time saw significant declines in the Warning performance level from 50% to 15% and an increase in the Proficient/Advanced performance level from 19% to 35%. In 2001, the 8th grade results at BBCS in Mathematics were above state averages in both the Warning (31%) and in the Proficient/Advanced performance levels (34%) The participation rates of 8th graders on the MCAS tests was 100%.

A cohort analysis of Benjamin Banneker MCAS results in English Language Arts revealed that from 1998 to 2001 the percentage change of students who received a score of Proficient/Advanced increased by 39% while percentages of students scoring in the Warning performance level declined by 20%. However, scores in mathematics remained stable.

Important Note

The Benjamin Banneker Charter School has been serving students since the 1996-97 school year when the school first opened as a charter school. Beginning in the spring of the third year of its charter, a school may apply for renewal of its charter for another five-year term. Specific guidelines for renewal have been set forth in the Application for Renewal of a Public School Charter. BBCS submitted its Application for Renewal with the Department of Elementary and Secondary Education and as part of that process underwent a four-day renewal inspection. In January 2001, the BBCS charter was renewed with conditions. These conditions include: defining the relationship between the board of trustees and the school; implementing the school's academic program; aligning and reporting student performance data on internal and external assessments; and improving student performance on the MCAS. In May 2001 BBCS responded to the conditions set forth by the Department by revising its Accountability Plan and Action Plans.

Panel Reponses To The Key Questions

KEY QUESTION 1: does The School Have A Sound Plan For Improving Student Performance?

Yes. The action plan and the accountability plan together constitute a "sound" school improvement plan that contains goals, strategies, timelines, responsible parties, progress indicators and measurement tools. The school's update of the plan this year has secured buy-in for improvement efforts among faculty, administrators and board members.

A. Has the school analyzed appropriate data and program information to accurately identify the gaps in student performance and determined why those gaps exist?

Yes. The school has done extensive analysis of student assessment data. Under the leadership of the Interim Executive Director this year, the Curriculum Director reviewed an analysis of 4th and 8th grade MCAS results. This new analysis revealed that grade 4 MCAS results in ELA and math were below state averages from 1998 to 2001 and declined slightly during the same period. However, 8th grade MCAS results from 2000 to 2001 in ELA and math increased.

MCAS item analysis for 4th grade ELA revealed that students scored poorly on questions about main idea, drawing conclusions and descriptive writing in addition to basic literacy skills such as vocabulary development. In math, students had difficulty overall in addition to basic skills and higher order tasks such as patterns, relations and functions, geometry, statistics and probability. In interviews, the Curriculum Director described the level of detail she undertook in analyzing MCAS items. She reviewed test items, looked for patterns in student responses, analyzed learning standards with end of year internal assessments, and compared the MCAS results with that of the Stanford 9. Details of this analysis are found in a notebook made available to the panel review team.

In addition to the MCAS, BBCS has a school-wide internal assessment system. This system assesses student performance three times per year --- referred to as "pre" "mid" and "post." BBCS contracted with a consulting firm to assess the degree of alignment between their internal measures and the MCAS. In ELA, a comparison of the open-response questions on the MCAS and the BBCS internal assessments revealed that the questions on MCAS are significantly more challenging than those on the current internal assessment system. As a result, the school is revising its ELA internal assessments to be as challenging as those questions on the MCAS. In math, the assessments that accompany the school's math program Trail Blazers (K-5) and MathScape (6-8), and the standardsbased curriculum are used for BBCS internal math assessments because they are aligned with the Massachusetts Curriculum Frameworks.

Teachers have ongoing opportunities to work with both internal and external assessment data in cluster team meetings, content area meetings and grade level meetings. In interviews, teachers were able to articulate how they analyze data and use it to change instruction. For example, teachers in cluster meetings analyzed the academic goals in the school improvement plan and compared that with the student needs identified by their more recent analysis of MCAS results. In a teacher focus group, one teacher explained that previously the emphasis in math was solely on number sense and computation. Based on the updated MCAS data analysis, the school has adjusted improvement efforts to include those other areas of need.

Teachers and administrators have also made strategic curricular and programmatic decisions based on data analysis. For example, analysis of MCAS math scores revealed that students were not performing well overall. Therefore, Trail Blazers (a K-5 math series) and MathScape (a 6-8 series) were adopted because they are more closely aligned with the Massachusetts Curriculum Frameworks. Also, as a result of reviewing the data, the school's literacy program was redesigned to include guided reading in K-8 in addition to an emphasis on phonics in the earlier grades and an emphasis on higher order thinking in the later grades. Finally, as a result of poor MCAS scores in the 4th grade, the after school MCAS program now accepts 3rd graders.

Teachers and administrators at BBCS clearly articulate their understanding of the reasons for poor student achievement. The Leadership Report identifies three areas: curricula not aligned with the state frameworks, internal assessments not addressing the frameworks, and uneven implementation of the standards among faculty. During interviews, teachers acknowledged that in the past the curriculum was not always covered and lessons were not always standards-based. In addition, some staff mentioned teacher turnover as a reason for poor student achievement.

B. Does the plan set out specific improvement objectives that are grounded in the school's analysis of the reasons for poor student performance?

Yes. The improvement objectives are related specifically to the conditions set forth in the written charter school renewal decision developed by the Department of Elementary and Secondary Education in 2001 and the updated goals based on the school's most recent data analysis. In addition, BBCS added specific objectives for increasing parent involvement and improving school climate.

All of the objectives focus broadly on teaching and learning and are achievable and measurable. For example, the action plan states that faculty will "align the curriculum with state frameworks in math, ELA, science, social studies and technology." To date, ELA, math and technology are completed. Science revisions are in process and are on-schedule to be completed.

Many of the school improvement initiatives are related directly to increasing student performance and are tied to gaps in student performance. For example, the action plan for 4th grade ELA and math states that staff will "develop an instructional plan for helping students master relevant ELA and math skills." The instructional plan includes English skills such as vocabulary, details and descriptive paragraphs. The math plans include order of operations, geometric 3d shapes, right angles, measurement, data analysis, number sentences, fractions and patterns. Teachers have been meeting in cluster teams since December 2001 to work on these plans.

C. In order to accomplish each improvement objective, does the plan specify strategies that appear likely to lead to improved student results?

Yes. There are a number of strategies that are connected to the reasons for poor student performance. They include teacher meetings/teaming, curriculum revisions, standards-based teaching, internal and external assessments, parent involvement and MCAS plans.

In order to implement, monitor and collaborate on the school improvement strategies, teachers have ample opportunity to work in teams. The formal team structure consists of weekly mandatory faculty meetings, weekly cluster meetings and content meetings in addition to informal teacher collaboration throughout the day. During weekly faculty meetings, the entire staff meets to discuss school-wide topics such as the school improvement plan, new ideas and concerns. For cluster meetings, the school is organized in three clusters - K-2, 3-6, 7-8 - in order to provide opportunities for teachers to plan across grade levels. Teachers meet weekly in clusters to scrutinize the curriculum, track which standards have been taught, analyze data and discuss instructional practice. In an interview, a teacher noted, "The cluster work is very beneficial."

Teachers meet in content teams less frequently than the weekly all-staff and cluster meetings. Nevertheless, content meetings are an important part of the teaming structure. During content meetings, teachers look more specifically at the content curriculum across grade levels, research new materials and discuss classroom practice. In addition to the above formal team structure, teachers meet informally throughout the day. In interviews, numerous teachers talked about how they collaborate with other teachers on a daily basis.

According to staff, a lack of curricular alignment has been one of the causes of poor student performance. As a result, curricular revisions have been completed in math, ELA and technology. There are plans to revise science with preliminary work underway that includes incorporating more hands-on instruction into the curriculum. In addition, math and ELA teachers have incorporated research-based instructional programs into the curriculum. In math, teachers are implementing Trail Blazers (a K-5 math series) and MathScape (a 6-8 series). These math programs emphasize numeracy and basic skills in addition to problem solving. In literacy, there is an integrated school-wide plan that includes guided reading K-6, Scott Foresman Literature program 4-6 and Houghton Mifflin series 7-8. For students that need literacy intervention, BBCS has implemented a number of research-based programs. They include: Tellian, a K-1 phonics-based program, Reading Recovery for grade one and Orton Gillingham K-8, a multi-sensory approach for teaching written language in a direct, sequenced and phonetic way. Interviews with teachers and administrators indicate that they believe the curricular revisions and programmatic choices in literacy and math will result in increased student performance. One teacher noted, "The curriculum isn't nebulous, we know what we are teaching."

In addition to curriculum alignment efforts, there has been a strategic change in the ways that the curriculum is delivered. BBCS has adopted a school-wide standards-based approach to lesson delivery. Lesson plans include setting clear objectives, direct instruction, teacher modeling and student practice. The Curriculum Coordinator reviews lesson plans, and she and the executive director drop into classrooms frequently. In interviews, teachers indicated that they feel supported in the classroom by the availability of the Interim Executive Director, Curriculum Coordinator, the Directors of Support Services and Title I, and through collaboration with peers in cluster and content meetings. Moreover, teachers were able to articulate the curriculum revision process, reasons for selecting the literacy and math programs and how they should be delivered classroom instruction.

Classroom observations revealed evidence of standards-based instruction. Panel review members observed a hands-on science lesson, a math lesson using manipulatives, and direct instruction in a lower grade literacy class. In follow-up interviews, teachers were able to discuss their instruction in relation to the curricular revisions and the overall goal of raising student achievement.

In order for teachers to upgrade their skills, the Interim Executive Director and leadership team have begun to make strategic changes regarding professional development. According to interviews, past professional development was not connected to school improvement efforts. Rather, it was driven by individual teacher's needs and desires. Although current professional development is still menu-driven, offerings such as reading recovery, MCAS and math skills workshops are connected to the school's improvement efforts. Professional development is moving towards more whole-school initiatives. This summer, teachers will participate in a series of workshops sponsored by Research for Better Teaching (RBT), a whole-school professional development program. In interviews, the Interim Executive Director was clear about her vision for professional development but acknowledged that the strategic changes would be phased in.

BBCS has made changes in the ways that it reports and uses student assessment data. As stated previously, an analysis of the internal and external assessments revealed needed changes in the internal assessments themselves, curriculum development and instruction. Moreover, teachers have ongoing opportunities to review student assessment data in cluster and content team meetings. In order to track student progress, BBCS has developed the Individual Student Success Plan (ISSP).

The ISSP is divided into four areas

• student assessment data
• identified individual weaknesses
• resources and strategies for assistance
• evaluation of progress.

Teachers are required to update the ISSP monthly. The ISSPs are housed in folders in the student's homeroom and will eventually be available on the computer. To date, student ISSP in grades 4 and 7 have been completed.

BBCS has instituted a number of strategies targeted specifically at MCAS preparation. There are mock MCAS exams for students in grades 3-8. Grades 3-4 have after-school MCAS preparation, and students in grades 4 and 8 have in-class MCAS preparation. When asked how students feel about the MCAS, one student said, "We are prepared because of the mock MCAS."

Parent involvement is one of the goals of the BBCS school improvement plan, and the school has begun a number of initiatives to get more parental support. Currently, there are monthly parent meetings, special workshops on topics such as MCAS and technology and a homework hotline for parents to access information about the school. The Board of Trustees has recently created a committee on parent involvement to support the school. Parents interviewed cited the ways that they were personally involved with the school. Despite the fact that there are multiple opportunities for parent involvement, they acknowledged the need for more parents to become involved.

D. Are the school's written improvement planning document (s) clear and specific enough to guide their implementation of planned improvement initiatives?

Yes. The school improvement plan consists of the accountability plan and the action plans. Together these two documents contain goals, expectations, strategies, timelines, responsible parties, progress indicators and measurement tools. In interviews with a cross section of stakeholders and in classroom observations, it is evident that these two documents are clear enough to drive school improvement efforts. The next step would be to combine both documents into a single plan.

E. Was the School Improvement Plan developed through a process that will support its successful implementation?

Yes. The school has successfully updated existing improvement documents with more focused goals and strategies that still remain true to the intentions of the required improvement objectives. The process has created a shared understanding of and support for the plan among key stakeholders.

BBCS finds itself in an unusual situation. The key school improvement documents - the accountability plan and the action plans - were written by the past Executive Director and past Curriculum Coordinator. The action plans were a direct response to conditions for renewal put forth by the Commissioner of Education in the school's charter renewal report and therefore must be adhered to in order to maintain a valid charter. However, the focus of the current school improvement efforts are driven not only by the conditions set forth in the charter renewal report but also by the more recent and detailed analysis of student performance data, curriculum and instruction which has been undertaken by the school's interim leadership.

In order to familiarize staff with the school improvement plan, the Interim Executive Director dedicated the first two months of weekly staff meetings in the fall of 2001 to reviewing the official action plans and accountability plan. During these meetings, staff had significant input on how the plans would be implemented and what their individual roles in implementation would be. A more in-depth and current review of needs was also undertaken as described above in response to question 1A. These efforts successfully familiarized all stakeholders with the required improvement objectives and both refined and augmented these objectives based on new data analysis and program review.

The school's meeting structure, which allows for frequent communication regarding the plan, makes possible the continual refinement of its improvement strategies. When asked about their involvement with the school improvement plan, teachers were able to articulate their responsibilities in implementing the plan. Moreover, many teachers cited the accessibility of the administration regarding improvement efforts.

In addition to the BBCS faculty, the Board of Trustees has become more involved in the school's improvement efforts. Partly because of pressure from the state, the Board has increased its membership from seven to sixteen members. New board members that have expertise in curriculum and instruction assist the leadership team in plan implementation. Finally, in interviews, parents were able to state the school's focus in improving student achievement.

KEY QUESTION 2: Are The Conditions In Place For The Successful Implementation Of The Improvement Plan(s)?

Yes. The conditions for successfully implementing improvement initiatives appear in place, although there are concerns across stakeholders that the instability of leadership that has plagued the school in the past could potentially hamper future progress if permanent leadership is not secured. The interim leadership at the school has the confidence of the vast majority of stakeholders and has proven capable of undertaking significant change efforts.

A. Does the school have effective leadership and sound management?

In August 2001 the BBCS Executive Director resigned. An interim director was hired immediately and is currently assuming leadership for the school. There has been significant turnover of school staff. According to the staffing report requested by the Department of Elementary and Secondary Education for the panel review, more than half of the current staff have been at BBCS for 2 years or less.

In response to the need for stability in policy, the Interim Executive Director and the leadership team have begun to put systems in place to create greater continuity. For example, the teaming structure, curricular revisions, standards-based lesson delivery, reporting, use of assessment data, professional development, and the continuation of a leadership team are all systems that reflect sound school management that are likely to continue regardless of who is selected to be the school's new executive director.

School leaders communicate effectively with a range of stakeholders. The teaming structure and weekly meetings provide significant opportunity for staff to communicate with leadership and each other. In interviews, teachers' comments reflected their satisfaction: "There is good communication with leadership." "There is good communication with the curriculum director through leadership meetings." In interviews, parents stated that the school communicates with them through an "opendoor" policy, monthly meetings and newsletters. Finally, interviews with the Board revealed that they feel informed regarding the operations of the school.

School leaders are focused on the school's mission, and there is a clear vision for accomplishing the school's improvement goals. The majority of staff focuses on high expectations for students. In an interview, one teacher summarized the sense of expectations for students, "There is a school wide focus on achieving excellence for all learners." In interviews, students confirmed this sentiment, "This is a better school than I went to before. The teachers care how I do in school." One parent echoed similar thoughts, "My daughter has grown academically here."

Moreover, most interviews revealed support for and confidence in the leadership. One teacher commented, "The Director has classroom experience which helps when she visits classrooms." Another teacher noted, " There is now routine and structure." One parent remarked, "The Interim Executive Director has established stability and consistency." A board member said, "We have confidence in the Interim Executive Director's ability to guide the school."

B. Is there evidence that the school's faculty supports the planned improvement efforts?

Yes. Evidence from interviews and classroom observations suggests that the faculty supports the majority of planned improvement efforts. As previously stated, there is a common understanding of reasons for poor student achievement, and these reasons are being dealt with by focusing on particular aspects of curriculum, instruction and assessment.

The only school improvement goal that lacks broad consensus and support is improving school climate. This goal deals with student behavior issues and there is evidence from the teacher survey and interviews that behavior management procedures are not followed consistently. In interviews, the Interim Executive Director expressed that she is aware of this problem, stating that some of the tension stems from differing philosophies of behavior management and leadership style between her and the previous Executive Director. Some progress is being made with focus on improved classroom instruction and a five-step process that dictates how to manage classrooms. Nevertheless, there appears to be more work to be done around this school improvement goal.

While there is broad support for the plan and a clear understanding of faculty roles for implementation, a number of teachers cited their concern about consistency and stability of the school leadership. One veteran teacher indicated that it was difficult making the shift between school heads that have different leadership styles. Another teacher summarized what many teachers said, "School success will result from consistent leadership."

C. Is the school receiving adequate guidance and support from the district leadership?

Charter schools are independent of the district within which they reside. However, the Board of Trustees is legally responsible for the school's operation and is charged with providing guidance and support to administration and faculty

In response to the renewal conditions concerning the Board, the school has added nine members with special expertise in curriculum, instruction, fund raising, finance and outreach. Moreover, the Board has divided its responsibilities into sub-committees and is more available to provide specific assistance to the school and the Interim Executive Director.

In interviews, the Chairman and board members appeared well informed of the school improvement efforts, the specific challenges facing the school and the ways in which the Board could provide assistance. For example, when asked about the low MCAS scores in the fourth grade, board members were aware of the school's analysis and reasons for poor student performance.

Board members and school leadership express support for each other. The Interim Executive Director formally reports to the Board on a monthly basis. She communicates informally with members of the Board on a more frequent basis.

There is evidence that some teachers are dissatisfied with the Board. Specifically, some teachers feel that the Board responds slowly to important issues. For example, the Interim Executive Director was hired a week before school. And the national search for a permanent director was not launched until March 2002 -over a year after the previous executive director gave notice.

Conclusion

The action plan and the accountability plan together constitute a "sound" school improvement plan that contains goals, strategies, time lines, responsible parties, progress indicators and measurement tools. The school's update of the plan this year has secured buy-in for improvement efforts among faculty, administrators and board members.

The conditions for successfully implementing improvement initiatives appear in place, although there are concerns across stakeholders that the instability of leadership that has plagued the school in the past could potentially hamper progress if permanent leadership is not secured. The interim leadership at the school has the confidence of the vast majority of stakeholders and has proven capable of undertaking significant change efforts.

Appendix A
Team Members

Geri-Lyn Ajemian, Ed.D. Coordinator, Massachusetts Department of Elementary and Secondary Education, Malden, MA
Nancy Clair, Ed.D. Panel Chair, Consultant, Schoolworks, Beverly, MA.
Cindy McKay, Guidance Counselor, Foxborough Public Schools, Foxborough, MA.
Maureen Scala-Freeman, Teacher, Medford Public School, Medford, MA.
Maureen Wesolaski, Ed.D Regional Curriculum Coordinator Spencer East Brookfield Regional School District Spencer, MA

Appendix B
Evaluating School Performance
Detailed Schedule for Review Panel School Site Visit

The times specified on the following schedule may be adjusted slightly to align with the daily schedule and practices in each of the schools being reviewed.

Day 1

Day 2 : All activities take place in the school