District/School Administration Administration
The Massachusetts Board of Elementary and Secondary Education

Recommendation to Revise the MCAS Performance Appeals Regulations for Science and Technology/Engineering

At the June 2008 Board meeting, I presented you with the 2008 MCAS Performance Appeals Annual Report. As part of that report, I identified the need to amend the performance appeals regulations to add Science and Technology/Engineering to the appeals process. As you know, beginning with the class of 2010, state regulation 603 CMR 30.03(3) requires students to meet or exceed the state standard in Science and Technology/Engineering in addition to English Language Arts (ELA) and Mathematics as a condition for high school graduation.

To help inform the recommendation that I am bringing to you this month, I named an eleven-member working group that included three Board members (Jeff Howard, Ruth Kaplan, and AJ Fajnzylber) and eight local educators. A list of the members and their affiliations is attached. The working group met at the Department on July 31, 2008. Several Department staff also participated in that meeting, including Deputy Commissioner Jeffrey Nellhaus who facilitated the discussion. I appreciate the working group's thoughtful consideration of the issues, which was very helpful in shaping my recommendation to the Board.

I recommend that the Board vote at the September 10th meeting to amend the MCAS Performance Appeals Regulations, 603 CMR 30.05, to make available a performance appeal for eligible students with respect to the high school Science and Technology/Engineering MCAS tests. This would be in addition to the performance appeal that already exists for the grade 10 ELA and Mathematics MCAS tests.

This memo presents a summary of the current MCAS performance appeals process, the recommendations of the working group, and the rationale for the approach I am proposing. I would encourage the Board to adopt the amendments as emergency regulations, which means they would take effect immediately and be available to eligible students early in the school year. (The usual course of adopting regulations entails about a three-month delay until they take effect.) Under the Administrative Procedure Act, the emergency regulations would remain in effect for no more than three months, during which time we would seek public comment and then bring the amendments back to the Board for a final vote.

Background on MCAS Performance Appeals

Massachusetts public high school students in the Class of 2003 were the first graduating class required to meet the state's Competency Determination standard in ELA and mathematics as a condition for high school graduation. The MCAS Performance Appeals process, established by the Board in 2002, provides students with a fair and reasonable opportunity to demonstrate that they possess the required knowledge and skills to meet the academic standard required for graduation even though they have not passed the MCAS tests in ELA and/or mathematics.

District superintendents are responsible for filing performance appeals on behalf of students. A student must first satisfy the eligibility criteria, which includes taking the grade 10 MCAS test in the subject area of the appeal at least three times, maintaining a 95 percent attendance rate in the prior year and the year of the appeal, and participating in MCAS tutoring or other academic support services. Once the eligibility requirements have been satisfied, superintendents can submit either a cohort or a portfolio appeal for the student.

A cohort appeal compares the student's grade point average to the grade point average (GPA) of a cohort of at least 6 other students who took the same sequence of courses in grades 10 and 11 in that subject area. The students who make up the appellant student's cohort must have earned a passing score in the 220-228 range on the grade 10 MCAS test in the subject area of the appeal. To be granted an appeal, the appellant student's GPA must equal or exceed the median GPA of the comparison cohort, or fall within two standard errors of the mean GPA of the cohort. An Appeals Board comprised of public high school educators appointed by the Commissioner meets monthly to review cohort performance appeals and make recommendations to the Commissioner.

When it is not possible to create a cohort of at least 6 other students, superintendents may file a portfolio appeal of the student's current and/or cumulative work in the subject area of the appeal. To be granted an appeal, a student's portfolio must demonstrate a comparable level of performance to that of a student who has passed the grade 10 MCAS test in the subject area of the appeal. A panel of experts in ELA and mathematics reviews each portfolio and makes individual determinations in each subject area. Portfolio appeals are reviewed three times annually, typically in November, April, and June.

Based on the evidence presented and the recommendation of the MCAS Appeals Board, the Commissioner decides whether to grant or deny the appeal or request additional information from the superintendent. If the evidence indicates that a student has not yet met the minimum academic standard for the Competency Determination, the student is denied the appeal and encouraged to continue coursework in ELA and/or mathematics in order to meet the standard. While most of the graduates in the classes of 2003-2007 (roughly 301,000 students) met the standard by passing the grade 10 MCAS tests or retests, more than 2,800 students have earned a Competency Determination through the MCAS Performance Appeals process, including approximately 335 students who earned it through appeals in both English language arts and mathematics.

The recommended changes to the appeals regulations will have no impact on the current appeals process for English Language Arts and Mathematics, other than to clarify that a student granted a performance appeal in ELA or Mathematics still needs to demonstrate proficiency or complete an Educational Proficiency Plan in order to meet the Competency Determination standard.

Working Group's Recommendations

The working group first reviewed the current appeals process and eligibility requirements that all students must meet in order to file an appeal. Next, the group was asked to focus specifically on the eligibility requirements to file an appeal in science only. We do not plan to change the method for determining whether students have met the passing standard, except for the fact that we will be using grade point averages from one course (the course associated with the test the student failed) rather than two courses as used by the current system (i.e. grades 10 and 11 English and/or mathematics).

Accordingly, the working group's discussion focused on the number of MCAS test attempts a student must complete in Science and Technology/Engineering before a student becomes eligible to file an appeal. After a thoughtful discussion about whether students should be required to take one or two science tests to be eligible to file an appeal, the working group recommended that students should be able to file an appeal in science after taking one test and not passing.

I concur with the working group's recommendation that we implement a "one test attempt" eligibility requirement for students who wish to file an appeal in Science and Technology/Engineering. My recommendation is based on the following rationale:

Rationale for "One Test" Eligibility Requirement to File a Science Performance Appeal

1. ELA and Mathematics are Cumulative Tests Whereas Science Tests are End-of Course Tests
   The Massachusetts Curriculum Framework standards in ELA and mathematics are cumulative in nature. Each successive grade level's standards reinforce and build upon the standards from earlier grade levels. Accordingly, while the grade 10 MCAS ELA and Mathematics tests focus primarily on grade 9 and 10 standards, assessment questions assume students have acquired knowledge and skills taught in earlier grades. Similarly, ELA and mathematics learning standards for grades 11 and 12 reinforce and build upon previous grade level standards in those areas. Thus, students who do not pass the ELA and/or Mathematics tests in grade 10 can strengthen their knowledge and skills in those areas through coursework in grade 11. Additionally they have two opportunities to pass an equivalent test to the one they failed in grade 10 during the eleventh grade. Therefore, current regulations require that students make three attempts to pass the grade 10 tests before they are eligible for an appeal.

   By contrast, high school level MCAS Science tests are end-of-course tests. They are based on standards that are unique to each subject (physics, technology/engineering, biology, chemistry). The science standards in each subject do not reinforce and build on each other in the same way that ELA and math standards do from grade to grade. This means that students who fail a science test on their first attempt, but pass the course in that area, will likely take a different course and a different test (consistent with their most current course), if necessary, the following year. The working group thought it would be reasonable for a student to be eligible for an appeal before having to take a different test from the one they first failed. It should be noted that students whose appeals are denied would have to take a second test to be eligible for a second appeal. For example, if a student fails his/her Biology test in grade 9, and then files an appeal in grade 10 but is denied the appeal, he/she must take another test in grade 10 in order to be eligible for a second appeal. The second test could be in either the same area he/she failed the first time or a different area, consistent with the most recent course they took.

2. Demanding Testing Load at the High School Level

While the working group acknowledged some benefits of requiring students to take more than one MCAS test in science in order to be eligible for an appeal, the group noted that high school students and teachers are already experiencing a demanding testing load, including MCAS tests in English Language Arts, Mathematics, Science, and U.S. History, MCAS retest opportunities in English Language Arts and Mathematics, tests required for students with Educational Proficiency Plans, and PSAT, SAT, and AP tests. The working group was particularly concerned that the Department be sensitive to the testing burden on educators and students.
3. Impact on Students in Vocational-Technical Education Programs
   Requiring that students make two test attempts in science would create a greater challenge for students in vocational-technical programs than in academic programs. Most vocational-technical programs teach students academic coursework in scheduled "blocks". One block is equivalent to one-half year of coursework. Most students in vocational-technical programs take one block of science a year and, therefore, require two years to complete a full-year course. If we make two test attempts the eligibility requirement, vocational-technical schools will likely respond by requiring students to take two science blocks each year, and would likely constrain the amount of time students spend in vocational/technical coursework.

Additional Eligibility Requirements

In addition to taking at least one MCAS test in science to be eligible for an appeal, the working group recommended that students: (1) meet the same attendance requirement of 95 percent during the year that the appeal is filed and the previous year, and (2) be required to enroll in a science course during the year in which they file an appeal in science - as an MCAS appeal cannot be filed until MCAS results are available, appeals cannot be filed until the school year following the administration of the test.

Next Steps

Students in the class of 2010, who are high school juniors in the current (2008-09) school year, should have the option of an MCAS performance appeal in science available to them as soon as possible. For that reason, I propose that the Board adopt these amendments as emergency regulations at this month's meeting. Under the Administrative Procedure Act, emergency regulations take effect upon adoption, a period for public comment follows, and then the agency reviews the regulations and takes final action on them within three months. The regulations expire at the end of three months if the agency does not take further action.

I recommend that the Board vote this month to adopt the proposed amendments to 603 CMR 30.05 as emergency regulations. We will invite public comment on the amendments through October 31 and then we will bring them back to the Board at the November meeting for a final vote. Deputy Commissioner Jeffrey Nellhaus and General Counsel Rhoda Schneider will be at our meeting to answer any questions you may have about the proposed amendments or the regulatory process.

   Performance Appeals Working Group Membership List