Family & Community Adult Education Adult & Community Learning Services
Adult and Community Learning Services

Assessment Update (Q& A # 2)

September 9, 2002

This is the second Assessment Update. (If you missed the first one, you can find it on the ACLS website at www.doe.mass.edu//acls/assessment, under Assessment Policy Update, #4 from August 9). The Assessment Policies and Procedures Manual is also on the website. Please read these if you haven't-they contain helpful information for administering and scoring the new assessments of TABE, BEST and REEP.

If you have other questions or comments on the assessment policies and procedures, please address them to Jane Schwerdtfeger at ACLS, at janes@doe.mass.edu or at (781) 338-3855.

BEST TEST-IMPORTANT NOTE!

The BEST test has two Forms, B and C. When scoring the BEST test, Form B, the totaled score of answers form the test is the scale score, and this is the score you would enter into SMARTT. When using Form C, however, the totaled score of answers must be converted into a scale score before you enter it into SAMARTT. To do this, please go to Appendix D of the Test Manual (pages 81-83). Follow the directions with the conversion chart on these pages to determine the scale score for Form C tests. This conversion is required so that both Forms of the test are equivalent.

QUESTIONS

Can I administer the TABE test to my students in the classroom during class?

Any of the tests, whether the TABE, BEST or REEP, should be administered in a quiet place where the student who is taking the test will be disturbed as little as possible. You should not administer the test during a regular class, unless all students are taking the test at once, such as the REEP test, which is designed to be administered in a class setting. This can be a challenge for programs that are short on space, but do the best you can with your own situation.

This question concerns accommodations that are made for students taking the TABE test, and that students must have a "documented formal diagnosis of a learning disability." Most students in Corrections settings (and others) do not have any documentation of this sort, though they have learning disabilities. What can we do?

We realize that very few students have this formal documentation, and unfortunately without it we cannot give additional time for these students to take the test. The challenge is that we cannot know for sure what learning disability that a student has without a relatively rigorous evaluation. The PAWG II, which will be convening in January 2003, will be looking at how learning disabilities affect adult learners and educational gain, and make recommendations for improving this problem.

Can I enter student intake information into SMARTT before I test the students for their initial assessment for reporting educational gain?

Yes. In SMARTT, the intake information is separate from the goals and assessments. Students can be enrolled in any class without the initial assessment for reporting educational gain, but attendance cannot be entered until students have been given the initial assessment. Once that is entered, the actual attendance can be entered, even when it occurred prior to the assessment date.

Can student goals be changed during the Fiscal Year?

Yes. As you have probably experienced, student goals do change over the year, as s/he masters some goals and wants to move on to mastering others. As in the case of a student who wants to get their GED, there are several smaller goals usually wrapped up in achieving that one goal. In SMARTT, you can enter new goals as they arise, and note the goals that have been either completed or abandoned.

Can you change the student's primary area of assessment within the Fiscal Year?

No, you cannot change the primary area for assessment reported in SMARTT that you use to report educational gain. For example, you cannot switch a student's primary assessment area from reading to writing, or from reading to math, in a fiscal year. (In other words, you can't compare a reading test with a writing test, because you can't compare the scores to measure gain.) You would also need to use the same assessment (though using a different form) for the initial, mid-year, and end-of-year reporting for educational gain. This is because the scores a student gets on one test--the BEST for example--can't be compared or correlated to the test scores of the REEP. You can give these other assessments so long as they are not the ones you are reporting for educational gain in the primary area in SMARTT.

Are the student goals linked to their area of primary assessment?

Reporting educational gain and reporting student goals are two separate things, though they should be connected to the maximum feasible extent.

A student would want to select the primary assessment area that is linked to their goals. Many students want to get their GED, which involves all 3 primary areas - reading, writing, and math. It is up to the student and the counselor/teacher to determine which area to select for assessment throughout the year. In reporting educational gain for a student, though, you wouldn't want to link the assessments to student goals too closely. There should be some "wiggle room" to allow for the goals to change, and not affect the primary assessment area, since it can't change during the year. Within "Get a GED," a student has two or three areas to choose from in what their primary assessment area will be. The student's primary assessment area will be based on whichever is their priority, and you (and they) don't have to focus on the other assessment areas. For example, if they choose improve reading within their overall goal of "get their GED," then the primary assessment area would be reading, and you would administer the TABE test to them every four months during the fiscal year. If their goal is to converse more effectively in English, then you'd administer the BEST test to the student.

Should I give a student the TABE Locator separate from the Complete Battery test? Can I give the student the Locator and Complete Battery test back to back?

Yes. You can administer the Locator first for placement purposes. You must also use it to determine the appropriate level of the Complete Battery test to administer for the initial test to report educational gain. You cannot administer the Locator and the Complete Battery test back to back, but must wait to administer the Complete Battery test between 2 and 4 weeks after intake. (Please refer to the August 9 Assessment Update for more information on placement).

Please also remember that you won't be using the TABE Word List, as it is part of Level L, which the state is not using to measure educational gain. Also, you only have to administer one Complete Battery test (either Language, or Reading, or Applied Math) for the student's primary assessment area, and enter the score into SMARTT. If you choose to administer more than the one required Complete Battery test for the primary assessment area, this additional test score would be listed in the "Other" assessment category, listed with the primary assessment box.

Can I give all the TABE Complete Battery tests at same time to different students (if I'm not administering it all to one class)?

Yes, you can easily administer the same level of the same subject area test to a group of students (all level E, or M, or D, or A). You would, however, need to administer the Applied Math test separate from Reading test, for example, as there are different directions, and it would be disruptive to students who weren't taking the same test.

Is the Locator a timed, or un-timed test?

The Locator is un-timed. It takes about 30 to 45 minutes to administer. You need to give the entire Locator test in order for it to give you reliable results regarding which level of the Complete Battery test to administer. You cannot give only individual sections of the Locator test.

When a student moves up from one level of difficulty in the TABE Complete Battery tests, will the student scores go down? If they go down, how will the scores reflect educational gain?

If you are using Forms 7 and 8, the scores won't go down if you go from one level of the TABE test to the next higher level. The tests are equivalent, and are normed to make sure that the scores reflect a student's educational gain. In the event that a student's score goes down for legitimate reasons, ACLS will not record a subsequent lower score as a reduction in educational level; instead it will be recorded as "no gain."

Can teachers see (NOT grade, just see) their classes' REEP essay tests?

Yes, teachers can and should see the writings. Hopefully, the writings will help inform instruction.

Can teachers go over the REEP essay tests with their class (unscored copies, with the originals in a safe place to be graded)?

No, teachers should not show and go over the actual essay test with their students. (Just as students do not see their BEST, TABE, and TOEFL tests, we don't want to render the REEP prompts useless for the future.) The scores should be reported to the students and the teachers can discuss areas of strength and weakness.

For program staff administering and scoring the REEP writing test, ACLS and REEP staff have developed a "procedures and helpful hints" document that clarifies how to use the test. You can find this document on the ACLS website, listed as "Procedures for Practitioners Administering and Scoring the REEP Writing Assessment," under http://www.doe.mass.edu/acls/news.html

Can you clarify the two waivers to assessment policies that I've seen recently?

Waiver One: Teachers cannot administer and score primary assessments for his/her own students.

Rationale: It is not appropriate for a teacher to administer or score primary assessments in a "high stakes" environment for his/her own students. This policy is not reflective of a lack of trust; rather it is something psychometricians agree will ensure more reliable results.

We realize this is a difficult task for many programs to implement immediately, and have concluded that wee need to allow more time for programs to set up a system that works for them. Therefore, this policy will be transitioned into place between now and the end of February 2003. By March 2003, no teachers may administer or score tests for their own students. Please note: the REEP test will be administered by the classroom teacher but not scored by the teacher. (See Assessment Policies and Procedures Manual 1.8.3 Administering and Scoring Tests.)

Waiver Two: The class placement test should not be the same test used to measure educational gain.

Rationale: The initial test for educational gain should not be given during or shortly after intake, but should be given after the student had significant contact with the program and staff. It should be given when a student is comfortable in the environment of the learning center (i.e., after an intake and substantial orientation process or after class placement.)

Please note: We understand that programs might already have a process in place that uses the educational gain pre-test assessment as a class placement test. In this case, we will waive this requirement through October 31, 2002, but thereafter programs will need to implement procedures for testing within approximately two to four weeks after intake. (See Assessment Update from 8/9/02, page 3.)

Can you recommend some placement tools as options for assessing ABE students at GLE Level 0-1.9?

Each SABES Regional Support Center has samples of placement tests that programs could look at and adapt; you can find contact information at http://www.sabes.org/. They have samples of assessment tools from the Cambridge Community Learning Center (for ABE); and for ESOL, have assessment tools developed by the Framingham Adult Learning Center, Quinsigamond Community College, the Community Education Project, and SCALE. Marie Cora, SABES Staff Development (Assessment) Specialist, has developed an excellent list of relevant articles from Adventures in Assessment, which analyze assessment tools. The bibliography is listed below.

Adventures in Assessment

Briefly annotated bibliography of articles focusing on In-take, Placement, and Goal-Setting, by Marie T. Cora, SABES Staff Development Specialist

Volume 1: Getting Started

All of these articles are relevant in terms of setting goals and discussing tools and procedures for in-take and/or placement of adult students.

"Assessment Issues: Research and Practice" by Lauren McGrail: Provides a brief overview of alternative assessment tools for in-take, in-class, and end of cycle.

"Partners in Evaluation" by Johan Uvin: Primarily about program evaluation, but conducted by participants; good ideas for ways to determine what adult students think.

"Getting in Touch: Participants' Goals and Issues" by Lucille Fandel: Nice presentation of goal-setting activities.

"Read/Write/Now Adult Learning Center Assessment Adventures" by Janet Kelly: Discusses alternative assessment procedures in detail, including tools for initial contact; screening and placement; goals-setting; and progress.

"Down and Dirty Miscue Analysis" by Lindy Whiton: Miscue analysis described; this is useful as an additional placement and/or diagnostic tool.

"The Education Goals Assessment Packet" by Martha Germanowski: Setting goals in detail.

"Alternative Assessment: An Annotated Bibliography" by Don Robishaw, ed.: Brief overview of alternative assessment tools for goal-setting and progress.

Appendix: Samples and examples of actual tools.

Volume 2: Ongoing

All articles focus on ongoing assessment; there are many self-assessment tools described.

"Self-Assessment: Doing and Reflecting" by Paul Trunnel: Describes self-assessment tools useful for goals-setting.

"Three by Three by Four: Ongoing Assessment at the Community Learning Center" by Karen Ebbitt et al.: Goals-setting tools and procedures described.

Appendix: Samples of tools for setting goals and surveying skills.

Volume 3: Looking Back, Starting Again

The first section focuses primarily on reflecting on progress at the end of a cycle; the second section focuses on the cyclic nature of using final evaluation to inform the beginning of a new cycle.

"Assessment and Planning: Giving Students Ownership" by Amy Gluckman et al. Discusses processes of in-take and the review of goals previously set by adult students.

"The Whole-Person Approach in Math Assessment" by Mary Jane Schmitt and Helen Jones. Math assessment in detail including learner comfort; long and short term goals; and initial assessment.

Appendix: Samples of some tools for both initial and on-going assessment.

Volume 4

This volume covers several "getting started" and "ongoing" articles.

"Group Goal Setting Activities" by PECE Resource and Planning Guide: Provides specific tools and procedures for conducting goals-setting with a group.

"Empowering the Student Through Goal-Setting" by Susan Martin et al.: Describes goals-setting procedures including initial assessment, in-take, and on-going measurement.

"The Informal Reading Inventory" by Eileen Barry: This tool is useful as an additional placement and/or diagnostic tool (higher level).

"The ESL Classroom as Community: How Self Assessment Can Work" by Dulany Alexander: Useful procedure for aiding the goal-setting process.

Volume 5: Tale of the Tools

This volume focuses entirely on tools for use in on-going assessment; although not the focus of this annotated bibliography, I highly recommend this volume: the tools described are excellent. This article, however, is relevant to this document:

"The Case for Pre-Goal Setting" by Don Robishaw: Goal-setting is described as a strategy as opposed to a static activity; interestingly, the recent move to revise Massachusetts' Goal-Setting Process raises some of the same issues that Don did in his article from 1993.

Volume 6: Responding to the Dream Conference

This volume focuses on writers' reflections and use of the first 5 volumes of Adventures in Assessment. Dispersed throughout is commentary on the points of goal-setting and alternative assessment procedures and tools. Of relevance to this document's purpose:

"Affirmation for Pre-Goal Setting" by Anne Marie DeMartino: This letter is in response to Don Robishaw's article from Volume 5, described above.

Volume 7: Partnership Project

This volume focuses on teachers as researchers in partnership with one another. Many of the articles reflect on on-going classroom events and progress of learners; of special interest is some focus on examining adult students' self-esteem and motivation.

"Taking Time to Talk: Students and Teachers Setting Goals" by Marty Tassi-Richardson and Deirdre McLaughlin: Procedures and tools for the goal-setting process developed in response to dissatisfaction with those in use at that time at a particular program.

"Self-Assessment for the Beginner: A Goals Oriented Approach" by Rudee Atlas and Dan Wilson: Although the focus is really about the on-going assessment process, this article provides some ideas for helping students identify goals and monitor achievements through self-assessment at lower levels and in a non-threatening format.

"Bottoms Up: An Alternative Self-Directed Readiness Training Program" by Don Robishaw: A follow-up article to Don's earlier one on the Pre-Goal Setting Process (Volumes 5); the program described is really for the on-going classroom, but still provides good ideas for making the goal-setting process more meaningful/valuable; some points could be adapted so that they could be incorporated into initial processes.

"Authentic Assessment in the Workplace" by Debbie Tuler: Describes the start of the program; discusses initial assessment tools/procedures, initial selection process, curriculum/materials development; includes descriptions of an orientation and interview process.

Volume 8

Covers a range of articles that focus on the influence/affect of alternative assessment on topics from education reform to program management to learner involvement.

Volume 9

The articles focus on examining how teachers (and students) assess continually in their classrooms.

"Assessment in the ESOL Experience" by Elizabeth Santiago: Discusses alternative and commercial tools for placement and initial assessment.

"Developing a Native Language Literacy Program" by Michelle Brown: Describes the quest by this program to find/develop an appropriate diagnostic tool for adults struggling with reading/writing skills in their own language.

"Learning from Experience: The Native Language Literacy Screening Device" by Deborah Mercier-Cuenca: This tool is reviewed; ordering information is provided.

Volume 10

Volume 10 covers articles discussing ways that practitioners try to integrate both commercial and alternative assessments into their instruction.

"The Haitian Multi-Service Center Experience" by Maria Kephallenou: Describes purposes of assessment from the different points of view of the participants (student, teacher, counselor, etc.); gives overviews of each, and includes overviews of intake/placement, and exit/entrance criteria; examples of tools are provided.

"Where's the EGAP These Days?" by Martha Jean: This article briefly updates this tool, whose acronym stands for Educational Goals Assessment Package; it is designed to capture a range of student's interests, goals, and some skill ability; examples of some of the tools are provided; the article does not give contact information, but I am aware that the practitioner-writer presently works with the YALD program.

"One Family Literacy Program's Assessment Story" by Sylvia Greene et al: The article describes their initial assessment and in-take processes and provides examples of tools; on-going and exit assessment procedures are also overviewed.

"The History of the BEST" by Moria Lucey; "Why I Think the BEST Isn't Good Enough" by Dulany Alexander; "The BEST is Workable, But It's Not the Only Choice" by Barbara Lippel-Paul; "An AmeriCorps Volunteer's First Impressions of the BEST" by Rachael Donnelly: I add these articles, whose titles are self-explanatory, for your information.

"What Counts? Assessing Computer Skills" by Kenneth Tamarkin: This article describes a tool and process designed to gauge the appropriate placement level for students; tool is also used to help measure progress.

"The TABE: Thoughts from an Inquiring Mind" by Cathy Coleman: The articles reviews and critiques this test.

Volume 11

These articles generally focus on how assessment is, or can be connected to teaching, learning, and program accountability.

"This is Only a Test" by Janet Isserlis: While this article does not focus on this document's purpose, Janet provides several web resources at the end including: 'Let's Get Started: An initial assessment pack for adult literacy programs' found at http://www.nald.ca/cl/getstarted/cover.htm; in addition, I highly recommend Janet's website for us all: I find it to be one of the most comprehensive adult/non-traditional education websites around; found at http://www.brown.edu/departments/swearer_center/literacy_resources/

"The More Things Change, the More They Seem to Stay the Same" by Maria Elena Gonzalez: The article reviews various initial assessment tools used over the course of several cycles of a program; examples of several tools are provided.

"Authentic and Learner-Centered Assessment in the Beginning ESOL Classroom" by Glen Cotton: While most of this article focuses on the practitioner-writer's research and subsequent classroom work, it does provide a brief description of processes for assessing learner purposes for learning English.

"Reflections on Meeting the Challenge of Assessment with Beginning Students" by Cheryl Gant: This article describes processes and tools (both commercial and alternative) that can be used to determine goals, and as a diagnostic at the lower ESOL level.

New Ways of Classroom Assessment by J.D. Brown, ed. as reviewed by Nancy Pendleton et al: Their review is quite positive and they note that the activities presented in the book include questions for pre-goal setting and self-assessment.

Volume 12

This volume focuses on teachers' and administrators' experiences working with standards-based reform initiatives.

"To TABE or Not to TABE: One Agency's Options" By Bernie Driscoll: This article describes an agency's alternative placement tool for students of math; the math tool, which does not appear appropriate for very low levels, is provided.

Volume 13: Meeting the Accountability Challenge

All the articles focus on practitioners' and administrators' efforts to meet state and federal accountability demands.

"Authentic Goal Setting with ABE Learners: Accountability for Programs or Process for Learning" by Sally Gabb: Sally provides a review of some of the articles focused on setting goals in Adventures in Assessment over the years; she also critiques the goal-setting process as it is presently carried out in Massachusetts.

"Quinsigamond Community College's Site-Specific Assessment" by Chris Hebert et al: The article describes the program's test for placing students at the SPL 7-10; the assessment is provided.

Volume 14: Examining Performance

All the articles in this volume discuss efforts to capture performance without the use of traditional or commercial tests.

"Assessing Oral Communication at the Community Learning Center" by JoAnne Hartel and Mina Reddy: The article describes the agency's test that can be used as a placement and diagnostic tool; examples of the tool are provided.

"So what is a BROVI, Anyway? And how can it change your (assessing) life?" by Betty Stone and Vicki Halal: While the tool is generally for on-going assessment purposes, many of the activities described can be adapted for use as a placement or diagnostic; examples of the tool are provided.

"A Writing Rubric to Assess ESL Student Performance" by Inaam Mansoor and Suzanne Grant: The REEP tool is described and the rubric is provided; as Massachusetts' practitioners now know, the tool is also used for placement purposes.

"Illuminating Understanding: Performance Assessment in Mathematics" by Tricia Donovan: This articles describes a process and rubric for working with performance assessment; while no specific tool or activity is presented, the approach is highly useful for practitioners in determining measures and diagnostic information when developing their own in-take and placement tools.