Family & Community Alternative Learning
1997 Test Results from
Massachusetts Charter Schools

Report Findings

As fourteen Massachusetts charter schools complete their second year of operation, and seven more finish their first year, results from standardized tests taken by charter school students are becoming available. More data will come over the months and years ahead. But there are now sufficient data to suggest answers to questions about the academic performance level of students entering charter schools and at least preliminary data about the academic gains those students have made.

What follows is a brief analysis of available standardized test results from charter schools, followed by an appendix comprised of testing data on each school. The report is organized around three questions:

• What is the academic performance level of students entering charter schools?
  

• Are charter school students making academic gains?
  

• How are charter schools measuring student performance?

While test results are the only indicators of performance analyzed in this report, they will not be the only indicators considered by the Department of Elementary and Secondary Education in assessing a charter school's performance. Standardized tests do not tell us everything about a school. Through reporting and formal site visits, the state will also gather data in regard to: academic performance (both academic -- state tests, performance-based measures and non-academic indicators -- such as drop-out rates and attendance); the organizational viability of the school, and the school's compliance with its charter and the attending state and federal laws and regulations.

These schools will need to be in operation a number of years before definitive statements about their success or failure can be made. Nevertheless, standardized test results offer comparable, credible measurement of the cognitive development of children. In the years to come, student performance will be measured based on the Massachusetts Comprehensive Assessment System (state tests aligned with the new state standards or Curriculum Frameworks). However, during this period when the state is changing its statewide testing system, these standardized test results offer valuable information about the charter school initiative.

1. What is the academic performance level of students entering charter schools?

Test results from the schools show that, with two exceptions, students who enrolled in charter schools on average, scored at or below district or national test averages (see Table 1 below, also see Appendix for more detailed information). This finding is consistent with the demographic data reported in The Massachusetts Charter School Initiative, 1996 Report.

Based on an analysis of available state, district and charter school data, it appears that 80% (12 out of 15) of charter schools that opened in 1995 have enrolled students that are average or below average in academic performance (Table 1 and Appendix). The students in ten of those schools appeared generally to be performing at or below their sending district averages (or national norms, when no district standardized test data was available). Two schools, YouthBuild Boston and Lowell Middlesex Academy Charter Schools, only enroll former drop-outs. Most of the seven charter schools that opened in 1996 have yet to report their standardized test scores.

Two schools, the Cape Cod Lighthouse Charter School in Orleans and the Francis W. Parker Charter School on former Fort Devens, despite open admissions processes, appear to have enrolled students that were, in general, performing above those of their respective sending districts (see Table 1 and attached Appendix).

Table 1: Academic Performance Levels of Entering Charter School Students

Notes:
NA = data not available.
NC = data not comparable.
* = Charter schools designed to serve former drop-outs.
-- = charter school testing data not yet available.
Massachusetts Educational Assessment Program (MEAP) scores for Boston Renaissance, Cape Cod Lighthouse, Francis W. Parker and SABIS International Charter Schools were reviewed, and generally corroborated with scores on national standardized assessments.

2. Are students in charter schools making academic gains?

Six out of the eight schools, from which adequate test data is available, appear to be making academic gains. These schools (Benjamin Franklin, Boston Renaissance, City on a Hill, Community Day, Lowell Middlesex Academy and SABIS International Charter Schools) opened in 1995 and have administered at least two rounds of testing (see Table 2 and Appendix).

As the oldest charter schools in the Commonwealth are only now approaching the completion of their second year of operation, these apparent gains should be viewed with care. Sample sizes in schools and grade levels are often small, so small changes in the composition of a given student pool could result in substantial shifts in school scores. Much more data will need to be gathered before any clear determinations can be made about the success or failure of any given school or the charter school initiative as a whole.

Perhaps the most considerable gains can be found at the SABIS International Charter School in Springfield. This school was one of the lowest performing in the district. Superintendent Negroni and members of the Springfield community invited SABIS International, Inc., a private educational management company, to take over the management of the school. Through this transition, the faculty and administration was completely changed while the 400 students who were attending the school remained. In the fall of the first year of this newly converted school, students tested (on average) below grade level in every subject in grades 2 - 6. Seven months later, these same students had on average increased 1.5 grade equivalent levels and the students in grades 2, 4, 6 and 7 were performing at or above grade level in every subject area and those in grades 3 and 5, while also making gains, were only slightly below grade level.

The academic gains are unclear in two of the eight schools. While students in both the Neighborhood House Charter School and Marblehead Community Charter Public School, began in 1995, administered two rounds of testing, and appear to be performing at or above grade level, the limitations of the available data do not allow for any clear conclusions in regard to academic gains. In the case of Neighborhood House, there are not enough comparable data from one year to the next on the same test and the same grade levels. In the case of Marblehead Community Charter Public School, there are insufficient data to draw any conclusions because the school used different versions of the same test from year to year. In 1995, MCCPS used the Stanford-8 and in 1996, it used the Stanford-9. The test publisher warns that, "schools selecting a new achievement test should avoid comparing scores from [Stanford-8 to Stanford-9] to draw conclusions about changes in performance...apparent drops (or increases) ... are not necessarily real decreases (or increases). Thus, in both cases, more data will need to be gathered using the same test in the years to come in order to validly evaluate the success of their respective academic programs."

Table 2: Are Charter School Students Making Academic Gains?

Notes:
* - Scores from these three Boston schools are from two different tests - the MAT-7 in the fall and the Stamford-9 in the spring. Because of this difference, these scores are not directly comparable. However, academic gains were assumed when: (a) students entered performing below grade level and performed at or above grade level in the spring; or (b) when comparisons relative to the district suggested academic gains.

Abbreviations:
CAT-5 = California Achievement Test of Basic Skills, Series 5
CBCPT = College Board Computerized Placement Test
IOWA = Iowa Test of Basic Skills
MAT-7 = Metropolitan Achievement Test, Series 7
Stanford-9 = Stanford Achievement Test, Series 9

3. How are charter schools measuring student performance?

Virtually all charter schools appear to be taking seriously their responsibility of being accountable for results. In addition to working to align their curricula with the state standards and participating in the state's testing program, charter schools have developed thoughtful school-level student assessment programs (see Table 3 for overview, also see the Appendix for more detailed descriptions).

Twenty of the twenty-two schools that have opened since 1995 have administered standardized assessments (Hilltown Cooperative, which opened in 1995, and Pioneer Valley Performing Arts Charter Schools, which opened in 1996, have not yet administered any standardized tests, but will be administering tests in the fall of 1997.).

Eighteen of these schools are utilizing some form of portfolio assessment (Table 3). Portfolios take many forms, playing a larger role in the assessment of some charter school students than others. Much like educators around the country, most charter schools are wrestling with how much information to gather, what the criteria should be in reviewing student work and how best to reliably score and report this data. Many of the charter schools are keenly interested in performance-based assessment and may develop models that could benefit other schools. Perhaps the most promising portfolio assessment model can be found at the Francis W. Parker Charter School which is part of the Coalition of Essential Schools. The school is still developing reliable methods for scoring and reporting data, but it has developed clear criteria for evaluating student work and has created a school culture that is self-reflective and has high standards.

Seven of the charter schools are working to refine competency-based promotion systems (Table 3). These schools reject connecting promotion with "seat-time." They believe that students must demonstrate that they can perform or understand the school's pre-defined skills or content required for promotion. For example, City on a Hill Charter School in Boston has established 11 competencies that they expect all students to know and be able to do before they graduate (such as being able to write a well-organized and interesting essay and give a 10 minute speech). Students are given opportunities to demonstrate their competency through an array of standardized and teacher-developed assessments as well as an end-of-the-year "juried assessment." Last spring, 72 members of the community were invited into the school to grade each of the students' ability to read, speak, write and compute. The results from this juried assessment were that only 30% of the ninth grade students met the competencies in the subject areas tested. Further, the board and faculty are committed to providing extra services in the summer or retaining unprepared students for an extra year or two so that when they receive a City on a Hill diploma they will have earned it.

Table 3: Student Assessments in Massachusetts Charter Schools

Notes:
* As mentioned in the text, portfolio assessment takes many different forms and may vary dramatically from school to school.

The reader should be reminded that this report does not purport to be a definitive statement on the academic progress of students in charter schools. These early results are more suggestive than authoritative. A more comprehensive study will follow after spring testing results have been compiled. Yet it can be concluded from the data thus far that on average the academic performance of students entering charter schools was at or below district or national averages; the early test results for charter schools look promising; and charter schools are taking seriously the responsibility of proving themselves worthy of public tax dollars by developing a range of thoughtful school-level accountability measures.

More data and a subsequent report will be available this fall. The state has administered third and tenth grade standardized tests this spring, and the schools have completed the administration of their own spring standardized tests. Once this data has been gathered a more complete report will be released.

Previous || Next
Table of Contents