Adult and Community Learning Services (ACLS)
Important CLAS-E Writing Assessment Information for Program Directors and Test Administrators.
October 5, 2011
Dear Program Directors:
Please read and share this important assessment information with TABE CLAS-E Test Administrators regarding their use of CLAS-E Retesting Guidelines when pre-testing learners.
Many program directors have expressed concern regarding the time and effort involved in following the CLAS-E Retesting Guidelines. I would like to explain why these guidelines are important, and to respond to questions about whether or not it would be equally effective to give a "retest," when indicated, at the optional mid-year test session.
The Retesting Guidelines were developed as a result of concerns raised last spring by many directors and test administrators about learners' pre-test scores that fell into the "NRS/SPL Plus" area. Many staff noticed that it was difficult for learners to show gain on subsequent tests when their scores fell into the "Plus" area. This concern was brought to the attention of the Center for Educational Assessment at UMass Amherst. Upon review, they confirmed that extremely low or extremely high scores for any CLAS-E level are much less reliable than scores in the middle range for a level. This is because each level of the CLAS-E is designed to meet specific content and skill targets that are progressive across levels. Learners who take a CLAS-E pre-test that is too hard or too easy will receive a final score that does not provide very much information about what that learner knows and can do relative to writing. Very low scores at a particular level suggest that a learner is not able to answer many questions at all at the administered level, and very high scores indicate that the student can demonstrate proficiency relative to the content and skills at the administered level and may perform well at another, higher level if one were administered.
When the learners' scores are far below or far above the content and skills tested at the CLAS-E level administered, their scores are unreliable. The tester's responses do not provide much information about the actual knowledge and skills at the learner's true proficiency level but rather are broadly indicative of performance on a too-hard or too-easy set of items. In particular, learners scoring into the "Plus" area would need to score almost as many items correct on the next highest level to show any meaningful gain because their original score was so inflated. For these reasons, the guidelines indicating when to retest up or down a level are important in order to get a valid and reliable score and an accurate understanding of the learners' knowledge and skills in writing.
One suggestion for making the retesting easier was to wait and administer the retest level indicated from the pre-test in December during optional mid-year testing. The Center for Educational Assessment and ACLS strongly discourage doing this, because any learning gains that learners make between the pre-test and mid-year test may be confounded by the pre-test score. This is to say that if a student's knowledge and skill improves between the pre-test and a mid-year retest, it is not known how those gains will affect the mid-year retest score, which may result in problems for demonstrating learner gains at post-test. Any re-test indicated is meant to be a pre-test, not a mid-year test. The pre-test is the benchmark for any gains the learner makes after that point.
ACLS and the Center of Educational Assessment strongly encourage programs to use the retesting guidelines and re-test when necessary. Given the level of concern expressed by staff regarding the requiring of retesting, however, ACLS will not require that retesting be done. While ACLS acknowledges the challenges presented by retesting, we urge program staff to use the guidelines so that learners' scores are as accurate as possible and most able to reflect the learner gains students and programs strive so hard to achieve. Some important points to reiterate about the retesting guidelines:
- These guidelines for re-testing apply to pre-testing only. They do not need to be used for mid-year or post-testing, or with scores rolled over form the previous fiscal year.
- If the score learners achieve on a pre-test indicates that re-testing is needed, test administrators should re-test as soon as possible. I understand that scoring the expository writing portion makes retesting a daunting task, but the sooner the retest is done, the more opportunity learners will have to make and show gains on subsequent tests, and not have those gains be subsumed in a delayed retest.
- The Guidelines should not be used to make decisions for other testing (e.g., whether or not to move levels for the post-test or in relation to scores rolled over from the previous fiscal year.) To determine when learners are ready to move to the next level test on mid-year or post-testing, take into account where in the range of the test level the scale score falls; if the score is in the upper range, the learner may be ready to post test in a higher level. As always, also use teacher judgment and knowledge of learners' performance in the classroom to determine readiness to tackle the next higher test level.
In Massachusetts, together we have worked hard to ensure that the use of standardized tests is about more than meeting a federal requirement. We have made a commitment to using testing as a tool to improve teaching and learning. While this commitment often demands time and resources that are in short supply, we hope that your commitment to this high standard will continue as you use the CLAS -E.
If you have any questions, please contact April Zenisky from the Center for Educational Assessment at UMASS Amherst at firstname.lastname@example.org. Thank you for your continued hard work.
With Literacy in Mind.
ABE State Director
Massachusetts Department of Elementary and Secondary Education