Adult and Community Learning Services (ACLS)

Revised Scoring for TABE Clas-E Practices and Certification

1. Difficulties in using the CLAS-E
   We can all agree that our use of the TABE CLAS-E has been extraordinarily challenging this year. I have heard many of your concerns personally about the costs of time and money, among other issues, and Jane Schwerdtfeger and Program Specialists have reported others' concerns to me. The TABE CLAS-E was vetted and approved by the US DOE for reporting ABE learning gains, and frankly, we are as surprised and as unhappy as you are about the difficulties we have undergone in using this assessment. Some of the factors that have caused statewide difficulties are:

   1. The Benchmark samples for the expository writing portion of the CLAS-E do not appear to be written by non-native English speaking ABE students. Therefore, many samples do not provide helpful or adequate comparison with which to evaluate our students' writing samples;
   2. The rubric score points are 0-3 or 0-4, and this structure of sequential scoring forces users to have a fraction if two scorers' scores are not exactly the same;
   3. The rubric is augmented by additional caveats separate from the rubric (i.e., "Notes to the scorer"), so it is difficult to attend to all the scoring information while scoring student tests;
   4. Technical support from CTB has been lacking in timeliness;
   5. CLAS-E issues notwithstanding, it is the first year Massachusetts has used this assessment, and we are used to evaluating student writing with an analytic (like the REEP's) rather than holistic rubric such as CLAS-E's.

   Most importantly and perhaps most damaging to us all, the test publishers' directions to us that two scorers must by consensus arrive at the exact same scores for the expository writing section has led to unreliable and unstable test scores. We are in the process of writing a letter to USDOE informing them of our concerns and lack of confidence in our CLAS-E test scores. Regarding our ongoing use of the CLAS-E, please refer to the attached revised instructions for scoring the CLAS-E expository writing portion of learners' tests. Test administrators and scorers, please follow these revised instructions for the rest of this fiscal year.

2. The CLAS-E Test Administrator Certification Process
   The process of scoring student writing samples to qualify for test administrator certification was complicated by a number of the points outlined above. Given the difficulties we have faced as a field in using this assessment, we have revised our criteria for qualifying CLAS-E test administrators.

   1. The original target for certification was 85% correct scoring in the 20 items in the test administrator certification packet. We have changed that target to 80% to accommodate some of the challenges test administrators had in scoring reliably. Please refer to the attached Revised TABE CLAS-E Policies for specifics.
   2. Seven of the 20 items in the test administrator certification packet do not have a completely clear-cut score. Scoring patterns on these seven items statewide similarly revealed almost equal divisions among two correct score responses selected by test administrators. For this reason, we instituted a "preferred score" and "acceptable score" rating for five of the seven items to acknowledge the difficulties in scoring. For two of the 20 items, either one of two scores qualifies as a correct response.
   3. The Curriculum and Assessment Coordinators are rescoring all of the test administrator certification packets and will email those who submitted a certification packet by the end of next week with their results.

   Test administrators who do not pass the CLAS-E certification will need to work with their regional Curriculum and Assessment Coordinator on their scoring, and not be able to score any CLAS-E expository writing assessments until they successfully complete their remediation. Please see the attached Revised TABE CLAS-E Policies for additional details.

   As we move forward with our use of the assessment, we are bringing our numerous concerns to the attention of USDOE and the CLAS-E test publisher. To those who have voiced their concerns, thank you for your part in helping to uncover the scope of these problems. We apologize for the difficult circumstances it has caused for all involved, program staff, students and ACLS alike.