Students & Families Adult Education Adult & Community Learning Services
Adult and Community Learning Services

Assessment and Recertification Information in Email from Anne Serino to Program Directors, 2-14-08

(This information was sent in an email to Program Directors on February 14, 2008 from Anne Serino.)

Dear Program Directors:

Please share this important and time-sensitive information about assessment with your staff. There are four assessment-related items covered in this letter:

1. Upcoming required BEST Plus Recertification needed by March 28, 2008
2. Upcoming required REEP Recertification needed by April 30, 2008
3. TABE Forms 7/8 Information update
4. MAPT Reading pilot items to be decreased

First, thank you to all who have taken part in assessing students this year. We know it continues to take time and requires careful attention on your part. It is important that we continue to administer and score the standardized assessments correctly. To that end, all test administrators and scorers must use the same test protocols. In order to maintain accurate scores statewide, please carefully review the information about the BEST Plus and REEP recertification processes below.

1. Upcoming required BEST Plus Recertification
   To ensure that all BEST Plus test administrators are scoring strictly according to the BEST Plus rubric, test administrators should re-familiarize themselves with the scoring process (known as recalibration) each time before they test.

   To ensure scoring accuracy in FY08, BEST Plus test administrators who recertified last year must recertify this year. Those whose certificates are valid through April 2009 do not need to recertify at this time. Recertification entails successfully completing Scoring Activity #2. To ensure successful completion of the activity, please review the BEST Plus Scoring Refresher Toolkit first.

   In order to recertify, you will need the following materials:

   • A 3-ring binder entitled, "BEST Plus Scoring Refresher Toolkit," with two accompanying DVDs or VHS videos
   • A white booklet entitled, "BEST Plus Scoring Refresher Workbook, Test Administrator's Edition"

   If you do not have these materials at your program, please contact your SABES Regional Curriculum and Assessment Coordinator to borrow a set.

   You must complete and submit Scoring Activity #2 to your SABES Regional Center between now and Friday, March 28, 2008. If you do not return your scoring activity by this date, you will not be able to administer the BEST Plus until you re-take the initial BEST Plus training, refresher, and scoring activity.

   By April 30, your Regional SABES Curriculum and Assessment Coordinator will review your work and inform you of your status. Scores will fall into three categories, based on the scoring system developed by BEST Plus test developers: "Recommended," "Acceptable," or "Unacceptable." Staff successfully completing the scoring activity will receive a letter and certificate valid though April 30, 2009. Staff that receive a borderline "Acceptable" or "Unacceptable" score in any one of the three rubric areas will have an opportunity to receive remediation in order to continue to administer the BEST Plus.

2. Upcoming required REEP Recertification
   In order to ensure that all REEP test scorers are recalibrated, and scoring consistently using the REEP rubric and anchor papers, all staff that score the REEP writing assessment need to recalibrate annually, and receive a certificate that they have gone through the recalibration process.

   On March 14, eight new homework essays will be mailed to you. You will be asked to score the essays and return the essay score sheet to your SABES Regional Center by Wednesday, April 30, 2008. If you do not return your homework by this time, you will need to re-take the REEP scoring training. Your Regional SABES Curriculum and Assessment Coordinator will then review your work and send out a certificate and/or letter to you by May 31, 2008.

   If you score the essays with 75% accuracy (6 out of 8 essays within one point of the actual score), your SABES Curriculum and Assessment Coordinator will send you a certificate certifying you through May 30, 2009. If practitioners score less than 75% accuracy (5 out of 8 essays), your SABES Curriculum and Assessment Coordinator will work with you on your essay scoring. If 4 essays or less are scored correctly, then you will need to re-take the REEP Scoring Training. You may not score until you re-take the training and again pass the REEP Scoring Homework essays.

   Here are some reminders about the REEP policies. REEP Training is divided into two parts: 1) REEP Administration Training and 2) REEP Scoring Training. If you haven't been trained yet in the REEP and you plan to administer or score the test this fiscal year, you'll need to take both before you may use the test. Please look for a schedule of trainings at www.sabes.org; or call your SABES Regional Curriculum and Assessment Coordinator. If you have other questions about the REEP assessment, please refer to the FY07 Assessment Policy and Procedures Manual for the REEP, found at http://www.doe.mass.edu/acls/assessment/.

3. MAPT for Reading pilot items to be decreased
   As you know, of the 50 items currently on the MAPT for Reading test, learners' scores are computed based on only 40 items. Currently, each learner taking the MAPT for Reading responds to 10 pilot items that do not count towards his/her score. These additional 10 items are needed so that we can build and improve the MAPT, using only the best items that have been carefully tried out on adult learners in Massachusetts.

   Since April 2007, when we began piloting 10 items on the MAPT, many teachers noticed their learners were taking much longer to complete the MAPT for Reading. We responded to these concerns by analyzing the average item response times for learners at each level before and after we increased the number of pilot items. From that report, we know that learners are indeed taking significantly longer on average. Since we do not wish the length of the test to have any negative impact on learners' performance on the scored items, we will henceforth be piloting only 5 items on MAPT for Reading.

   Beginning February 19th, your students' scores will continue to be computed on the basis of 40 operational items, however, the number of pilot items embedded in the MAPT for Reading will be reduced from ten to five. This change will not impact tests that are in progress, and there will be no shutdown of testing in order to implement the shortened test structure.

   Please note that this change only affects the MAPT for Reading. This change to the MAPT for Reading will bring both tests-MAPT for Reading and MAPT for Math-to the same length in terms of number of total items.

4. TABE Forms 7/8 Information update
   The test publishers of the TABE, McGraw-Hill Contemporary, will be discontinuing Forms 7/8 of the TABE on March 31, 2008. Our state, however will continue to use Forms 7/8 for FY08 and most likely in FY09 as well. If you need to purchase any Form 7/8 materials, please do so before the end of March.

If you have any questions about these four topics, please contact Jane Schwerdtfeger at janes@doe.mass.edu

Thank-you,
With Literacy in Mind.
Anne

Anne Serino
State Director, Adult and Community Learning Services
Massachusetts Department of Elementary and Secondary Education