Massachusetts Comprehensive Assessment System

On the Desktop — August 30, 2019

News from Commissioner Jeffrey C. Riley & the MA Department of Elementary and Secondary Education

Update on Grade 10 ELA MCAS Essay Prompt

Dear Superintendents, Charter School Leaders, Assistant Superintendents, and High School Principals,

Today, an independent study was released examining students' performance on the English language arts section of the spring 2019 10^th grade MCAS test. The study, led by Stanford researchers, showed that while black students experienced a small difference in performance as compared to white students on the second day of the English Language Arts MCAS test, that small difference was within the normal variation of such differences on past MCAS tests. When viewed in the context of other MCAS tests, the researchers conclude that the small difference "is similar to effects, both positive and negative, observed in other MCAS settings."

Indeed, preliminary MCAS results show that the same percentage of students met the testing requirement for graduation this year as did last year. That is true both for students overall and for black students.

In March, I announced that DESE would not score an essay question that asked 10^th grade students to write from the perspective of a character who is insulting to a girl who is a runaway slave but nevertheless shelters the girl, as described in a passage from Colson Whitehead's novel The Underground Railroad. I also invited two researchers at Stanford Graduate School of Education to examine whether the presence of that question affected the performance of students on the parts of the test that came after that essay prompt. The essay prompt came on the second day of the test, and the questions that followed it included some that were there for field test purposes only and did not count toward students' scores. Only a relatively small number of scored items followed the essay prompt.

Stanford researchers Thomas S. Dee and Ben Domingue found a small difference in black students' performance after the essay question compared to the performance of white students who scored at similar levels on the first day of the test. However, the researchers also found that this small difference was within the range of differences between the performance of black students and white students on the later portions of tests from other grades and years.

The researchers note that these results may be due to the fact that they were focusing on only a small number of questions, given that there were only a small number of scored questions after the essay question, and small numbers may introduce such variability. They note that their findings suggest that the essay question influenced the performance of relatively few students.

Although the study did not find major differences in the performance of black students as a group, a small number of students may have experienced a greater impact. In recognition of this possible effect, and out of an abundance of caution, DESE has decided to take the following steps for all students, regardless of race or ethnicity:

1. The Department will grant exceptions to the required minimum test score for students whose performance prior to the essay prompt was on track to meet the test requirement for graduation (equivalent to 220 points on the legacy MCAS) but who struggled on the last set of questions after the voided passage. (Based on a preliminary analysis, fewer than 100 students — out of 70,000 tested — fit this description.)

2. The Department will allow the students who took this test to retake it in November 2019 if they believe that the presence of the essay question impaired their ability to score high enough to qualify for a John and Abigail Adams Scholarship. Students have always been able to retake the 10^th grade MCAS test if they do not pass it on the first try. The differences this year are that students who met the graduation test requirement would be allowed to take the retest, and they would be allowed to qualify for an Adams scholarship on that retest.

3. The Department will allow students who took the 10^th grade MCAS in spring 2019 to be considered for an MCAS performance appeal if they take the English Language Arts MCAS test twice without success, instead of three times.

The Department also continues to look for ways to improve how the agency serves students, schools, and districts. The Department will continue to improve the test development process, in which we collaborate with educators, by:

• continuing to ensure that DESE's Bias and Sensitivity Review Committee reflects diverse backgrounds and viewpoints, and setting that diversity as a goal for our Assessment Development Committees, too;
• developing additional written guidelines and policies to assist the review committees; and
• conducting additional training for committee members and DESE and contractor staff.

We are taking the actions outlined here because we believe they are in the best interest of students, who continue to be our primary focus.

Sincerely,

Jeffrey C. Riley
Commissioner