[Massachusetts Department of Elementary and Secondary Education Logo] The Massachusetts English Proficiency Assessment Guide to Interpreting the Spring 2011 MEPA Reports for Schools and Districts June 2011 Massachusetts Comprehensive Assessment System [Massachusetts Department of Elementary and Secondary Education Logo] This document was prepared by the Massachusetts Department of Elementary and Secondary Education Mitchell D. Chester, Ed.D. Commissioner The Massachusetts Department of Elementary and Secondary Education, an affirmative action employer, is committed to ensuring that all of its programs and facilities are accessible to all members of the public. We do not discriminate on the basis of age, color, disability, national origin, race, religion, sex or sexual orientation. Inquiries regarding the Department’s compliance with Title IX and other civil rights laws may be directed to the Human Resources Director, 75 Pleasant Street, Malden, MA 02148 781-338-6105. © 2011 Massachusetts Department of Elementary and Secondary Education Permission is hereby granted to copy any or all parts of this document for non-commercial educational purposes. Please credit the “Massachusetts Department of Elementary and Secondary Education.” Massachusetts Department of Elementary and Secondary Education 75 Pleasant Street, Malden, MA 02148-4906 Phone 781-338-3000 TTY: N.E.T. Relay 800-439-2370 www.doe.mass.edu [Massachusetts State Seal] Foreword The Massachusetts English Proficiency Assessment (MEPA) tests English language learner (ELL) students in four areas: reading, writing, speaking, and listening, as mandated by state and federal law. The MEPA tests are an important part of our state’s effort to improve teaching and learning for English language learners. It is essential that teachers, administrators, and parents understand the MEPA testing program and how it can help students succeed. The Department of Elementary and Secondary Education (“the Department”) is committed to working in partnership with schools and communities to improve the academic achievement of all public school students in Massachusetts. This guide explains results and other information contained in the Roster of Student Results, the Preliminary Participation Report, the Preliminary Results by Year of Enrollment in Massachusetts Schools report, and the Progress Report for the spring 2011 MEPA tests. The guide is intended to assist you in reviewing your school and/or district results. MEPA results are reported according to five performance levels: Level 1, Level 2, Level 3, Level 4, and Level 5. Information that will help educators interpret the meaning of these performance levels is included in this guide. If you have questions after reviewing this guide and your reports, please contact Student Assessment Services at 781-338-3625. Table of Contents Foreword I. Document Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 II. Overview of MEPA Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Measures of Performance on the MEPA Tests . . . . . . . . . . . . . . . . . . .3 MEPA Performance Level Descriptors . . . . . . . . . . . . . . . . . . . . . . .3 III. General Guidelines for the Use and Interpretation of MEPA Reports . . . . .8 Use of MEPA Student-Level Results . . . . . . . . . . . . . . . . . . . . . . . .8 Understanding Scaled Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 State Regulations Regarding Confidentiality of Student Records . . . . . . . 9 IV. Understanding the Roster of Student Results . . . . . . . . . . . . . . . . . . . 10 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Information Included in the Spring 2011 MEPA Roster of Student Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 V. Understanding the School and District Preliminary Participation Report and Preliminary Results by Year of Enrollment in Massachusetts Schools . . . . . . . . . . . . . . . . . . . . 14 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 School Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Minimum Number of Students Needed to Report Results . . . . . . . . . . 14 Information Included in the Spring 2011 MEPA Preliminary Participation Report . . . . . . . . . . . . . . . . . . . . . . . . 15 Information Included in the Spring 2011 MEPA Preliminary Results by Year of Enrollment in Massachusetts Schools . . . . . . . . . . . . 17 VI. Understanding the 2011 Progress Report . . . . . . . . . . . . . . . . . . . . . 19 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Progress of Students Tested in Spring 2011 . . . . . . . . . . . . . . . . . . . 19 Students in Level 4 High and Level 5 in Spring 2011 by Number of Years in Massachusetts Public Schools . . . . . . . . . . . 20 Comparison of Performance Levels of Students Tested in Spring 2011 and Spring or Fall 2010 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Appendix MEPA Overall Raw Score to Scaled Score Conversion Tables . . . 23 I. Document Purpose The purpose of this guide is to support local educators’ use of district-, school-, and student-level reports of results from the spring 2011 administration of MEPA. In addition, this document can help school and district personnel communicate with parents and the public about the meaning of MEPA results. This guide describes how to interpret information provided in the following MEPA reports: • Spring 2011 MEPA Tests: Roster of Student Results • Spring 2011 MEPA Tests: Preliminary Participation Report • Spring 2011 MEPA Tests: Preliminary Results by Year of Enrollment in Massachusetts Schools • Spring 2011 MEPA Tests: Progress Report These reports contain information that is valuable to local educators in their efforts to improve curriculum and instruction and to serve the academic needs of English language learner (ELL) students. For guidance on how to use MEPA results to make classification decisions, please see the Department’s publication Guidelines on Using MEPA Results to Plan Sheltered English Immersion (SEI) Instruction and Make Classification Decisions for Limited English Proficient (LEP) Students, available at www.mass.edu/ell/guidance_laws.html. II. Overview of MEPA Tests Title III of the No Child Left Behind (NCLB) Act of 2001 requires that states annually measure the performance of ELL students and their progress toward acquiring English language skills in the domains of reading, writing, listening, and speaking. In addition, Chapter 386 of the Massachusetts Acts of 2002 (known as Question 2) requires ELL students in Massachusetts to participate in assessments of English language proficiency. The MEPA program complies with these federal and state assessment requirements. MEPA results are used to • help determine the level of English proficiency of ELL students; • measure student, school, and district progress toward meeting the state’s learning standards as detailed in the English Language Proficiency Benchmarks and Outcomes for English Language Learners (www.doe.mass.edu/mcas/mepa/benchmark.html); • improve student achievement and classroom instruction by providing diagnostic feedback regarding student acquisition of knowledge and skills. The Massachusetts Department of Elementary and Secondary Education defines an ELL student as “a student whose first language is a language other than English and who is not able to perform ordinary classroom work in English.” All ELL students in grades K–12 educated with Massachusetts public funds participate in MEPA, including • students enrolled in public and charter schools, • students enrolled in educational collaboratives, • students enrolled in private schools that receive public funding for special education (including approved and unapproved schools within and outside Massachusetts), • students who receive educational services in institutional settings, • custodial students of the Departments of Children and Families and Youth Services, • students with disabilities. The MEPA test consists of two separate assessments. The MEPA-Reading/Writing (MEPA-R/W) is a written test that assesses reading and writing knowledge and skills. All ELL students in grades K–12 with very few exceptions were required to participate in the MEPA-R/W, which was developed for ELL students in five grade spans: K–2, 3–4, 5–6, 7–8, and 9–12. In grades 3–12, the separate reading and writing tests consist of three test sessions, each of increasing language complexity. Each student participated in two sessions of both reading and writing. Schools made separate decisions about which two sessions a student would take and were instructed to consider prior MEPA and MCAS ELA scores, results of locally administered reading and writing diagnostic assessments, and classroom observations of the student to evaluate which two sessions best matched the student’s needs. In grades K–2, students were administered either Level A or Level B tests in reading and writing according to their level of English language proficiency. The Massachusetts English Language Assessment-Oral (MELA-O) is an observational assessment that evaluates listening (comprehension) and speaking (production) skills in English. All ELL students in grades K–12 are required to participate in the MELA-O with very few exceptions for students with certain disabilities. Qualified MELA-O trainers and/or administrators used the MELA-O Scoring Matrix to assess ELL students’ listening and speaking skills by observing the students as they participated in everyday classroom activities. Measures of Performance on the MEPA Tests Performance on the MEPA tests is reported according to • an overall scaled score that ranges from 400 to 550; • a performance level (Level 1, Level 2, Level 3, Level 4, or Level 5); • scores for each of the four learning areas (Reading, Writing, Listening, and Speaking) as follows: Reading Scaled score range 0–30 Writing Scaled score range 0–30 Listening Raw score range 0–5 Speaking Raw score range 0–20 Please note that the reading and writing scaled scores are designed to allow comparisons of reading or writing scores of students who took sessions 1 and 2 to those of students who took sessions 2 and 3. MEPA Performance Level Descriptors Each student’s overall results on the MEPA tests are reported based on five levels of performance. The performance level definitions for grade span K–2 vary slightly from the performance level definitions for grades 3–12. MEPA performance level definitions for both grade span K–2 and grade spans 3–12 are provided on pages 4–7 of this guide. Grade Span K–2 A student at Level 1 cannot yet communicate in English, and errors almost always interfere with communication. Comprehension is demonstrated either without words, through a few basic English words, or in a language other than English. A student performing at this level typically • recognizes only a few letters of the alphabet and reads only a few simple words, with help; • writes only a few letters of the alphabet and may attempt to write words that are mostly incorrect; • speaks using only a few English words with frequent errors, and cannot be understood easily; • understands only a little spoken English. A student at Level 2 communicates using simple written and spoken English at school, with errors that often interfere with communication and understanding. A student performing at this level typically • recognizes many letters of the alphabet and reads a few simple words, with help; • writes some letters of the alphabet and a few basic words, with frequent errors; • speaks English using basic words and short phrases, and is generally difficult to understand; • recognizes some basic spoken words and phrases in English, with frequent repetition and explanation. A student at Level 3 communicates using basic English at school, though errors sometimes interfere with communication and understanding. A student performing at this level typically • reads and understands many common words and most letters of the alphabet, including some grade-level academic language; • writes commonly used words and simple sentences, with some errors; • speaks English using common words, including some grade-level academic vocabulary, and basic grammar and sentence structure, with some errors; • understands most basic spoken English, including some grade-level academic vocabulary, with some repetition and explanation. A student at Level 4 is generally fluent in English at school. Oral and written communication is mostly correct and usually understandable, with few or minor errors. A student performing at this level typically • reads and understands most common words, including many grade-level academic words; • writes short sentences and short texts, with few errors; • speaks English with basic fluency, using grade-appropriate words and sentences, with occasional errors; • understands most spoken English during classroom discussions, including grade-level academic vocabulary, with only occasional repetition and explanation. A student at Level 5 is fluent and communicates effectively in English across all academic subjects, with very few errors. Oral and written communication is correct and understandable. A student performing at this level typically • reads and understands most grade-level texts; • writes and edits grade-level texts with fluency, using standard grade-level English writing conventions; • speaks English with fluency, using grade-level academic vocabulary correctly; • understands spoken English, with little or no need for repetition or explanation. Grade Spans 3–4, 5–6, 7–8, and 9–12 A student at Level 1 cannot yet communicate in English, and errors almost always interfere with communication. Comprehension is demonstrated either without words, through a few basic words, or in a language other than English. A student performing at this level typically • reads only a few simple written words or phrases, with help; • writes only a few simple words and a few short sentences with errors; • speaks using only a few English words with common errors, and is not easily understood; • understands only a little spoken English. A student at Level 2 communicates using simple written and spoken English at school, with errors that often interfere with communication and understanding. A student performing at this level typically • reads and understands simple words, phrases, and a few simple sentences with help, but shows little awareness of features of written English; • writes one or more simple sentences with some understanding of purpose and audience, but shows little control of grade-level standard English writing conventions; • speaks using basic English words and phrases, and is generally difficult to understand; • understands some basic spoken vocabulary, phrases, and simple questions, with frequent repetition and explanation. A student at Level 3 communicates using basic English at school, although errors sometimes interfere with communication and understanding. A student performing at this level typically • reads and understands many common words and some grade-level academic vocabulary; can understand the main idea of some grade-level texts, and understands some grade-level features of written English; • writes and edits simple sentences and paragraphs to suit an audience, and uses basic grade-level vocabulary; shows some control of grade-level standard English writing conventions; • speaks using many basic English words and some grade-level academic vocabulary, creating original sentences, with some errors and pauses in conversation; • understands most spoken English sentences and questions, some basic grade-level academic vocabulary, and grade-level texts read aloud, with some repetition and explanation. A student at Level 4 is generally fluent in English at school, and oral and written communication is mostly correct and usually understandable, with few or minor errors. A student performing at this level typically • reads and understands most grade-level texts, including academic vocabulary and most grade-level features of written English; • writes and edits short texts with few errors using basic grade-level academic vocabulary, and shows basic control of grade-level standard English writing conventions; • speaks English with basic fluency, using grade-level words and sentences, with occasional errors; • understands most spoken English during classroom discussions, with only occasional repetition and explanation. A student at Level 5 communicates effectively in English across all academic subjects, with few errors. The student shows control of standard English. Oral and written communication is correct and understandable. A student performing at this level typically • reads and understands most grade-level texts, including a range of academic vocabulary; • writes and edits texts of different lengths, giving details and descriptions to suit the purpose and audience, and shows a general control of standard grade-level English writing conventions; • speaks English with grade-level fluency, using academic language and descriptive vocabulary in conversations and classroom discussions; • understands spoken English during nearly all conversations and classroom discussions. III. General Guidelines for the Use and Interpretation of MEPA Reports Use of MEPA Student-Level Results The MEPA results assist schools and districts in evaluating students’ progress toward achieving the state’s standards identified in English Language Proficiency Benchmarks and Outcomes for English Language Learners. MEPA results are intended for use in guiding decisions regarding the planning of instruction for ELL students and the readiness of ELL students to perform ordinary classroom work in English without supplemental support. Massachusetts also uses annual spring MEPA results in the Annual Measurable Achievement Objectives Report, which describes the progress Title III districts are making in teaching ELL students. Understanding Scaled Scores Scaled scores are used to report student performance in a comparable manner across test sessions. The scaled scores allow student performance to be compared without regard to which pair of sessions was administered. The student’s overall MEPA scaled score is a combination of his or her scores on both the MEPA-R/W and the MELA-O. In order to calculate each student’s overall MEPA scaled score, a formula is used to map the total raw score to the overall MEPA scale. This formula takes into account how hard or easy the questions are for the test sessions that were administered. Please see the appendix, which begins on page 23 of this document, for the raw score to scaled score conversion tables. The 150-point scaled score range (400–550) allows for reporting the full range of item difficulty and for reporting five performance levels. The advantage of this scale is that it optimally reflects all available raw score points, regardless of the sessions completed by each student, while maintaining the distribution of observed student performance. The table on the next page shows the overall MEPA scaled score cut points for each grade span. Many of the scaled scores associated with the different levels vary across grade spans because the knowledge, skills, and abilities measured by the items vary from one grade span to the next. However, the scale for each grade span begins Level 1 at 400 and Level 5 at 500. These fixed values are used consistently for all grade spans to assist in interpreting student performance results across the grade spans. Spring 2011 MEPA Scaled Score Cut Points Grade Span Level 1 Level 2 Level 3 Level 4 Level 5 K–2 400 453 466 485 500 3–4 400 432 452 474 500 5–6 400 436 456 479 500 7–8 400 443 464 486 500 9–12 400 450 464 489 500 In order to receive an overall scaled score and performance level, a student must have complete scores in each of the four learning areas (reading, writing, listening, and speaking). If a student has an incomplete score in one or more of the learning areas, a non-participation code is assigned in place of a total MEPA score. Please note that students taking sessions 1 and 2 (or Level A in grade span K–2) cannot score at Level 5. Additionally, note that the overall scaled score range of 400–550 is not the sum of the score ranges for the four learning areas. State Regulations Regarding Confidentiality of Student Records The Massachusetts Student Record Regulations (603 CMR 23.00) and the federal Family Educational Rights and Privacy Act (FERPA) require that access to individual student results, including those provided in the MEPA Roster of Student Results and the Parent/Guardian Report, be restricted. These results are part of students’ temporary records, which contain all of the information that is not contained in the transcript and generally cannot be examined without consent from the parent/guardian or student, by anyone other than the student, his or her parents or guardians, and authorized school personnel. Superintendents and principals are responsible for maintaining the privacy and security of all student records. In accordance with these state and federal regulations, authorized school personnel shall have access to the records of students to whom they are providing services only when such access is required in the performance of their official duties. For more information on this topic, refer to the Massachusetts Department of Elementary and Secondary Education website at www.doe.mass.edu/lawsregs/advisory/cmr23qanda.html. IV. Understanding the Roster of Student Results Overview The Roster of Student Results provides results from the spring 2011 MEPA tests for all ELL students in a school in grade spans K–2, 3–4, 5–6, 7–8, and 9–12. The report also provides the most recent results from prior MEPA administrations, if available, for each student. This report shows how each student in a school • performed in each of the tested learning areas (reading, writing, listening, and speaking), • performed overall on the MEPA test. A sample taken from part of a fictitious Roster of Student Results is shown below. [Sample of a Fictitious Roster of Student Results] Information Included in the Spring 2011 MEPA Roster of Student Results The Roster of Student Results lists all students in the tested grade span alphabetically by last name in the left-hand column (not included in the sample above). The report contains a row for each student for whom your school submitted a spring 2011 answer booklet and/or for each ELL student enrolled in your school as reported in the March 2011 Student Information Management System (SIMS). If a student’s name was not available, “BLANK NAME” is displayed at the end of the Roster of Student Results. The student’s State-Assigned Student Identifier (SASID) is displayed in the column next to the student’s name (also not included in the sample above). Years in Massachusetts This column lists the number of years each student has been enrolled in Massachusetts public schools at the time of the spring 2011 test administration, according to March 2011 SIMS data. The number of years enrolled in Massachusetts public schools is reported as 1, 2, 3, 4, or 5+ (see table below). If this information was not available for a student, a (–) is displayed. Number of Years Definition of Enrollment Duration 1 Student is in his or her first year of enrollment in Massachusetts public schools. 2 Student is in his or her second year of enrollment in Massachusetts public schools. 3 Student is in his or her third year of enrollment in Massachusetts public schools. 4 Student is in his or her fourth year of enrollment in Massachusetts public schools. 5+ Student is in his or her fifth year or more of enrollment in Massachusetts public schools. Administration This column shows the administrations included on the report. Each student should have results for spring 2011 and may also have results for spring 2010, fall 2010, or spring 2009. If a student participated in more than one of these test administrations, and his or her records from each administration were able to be matched based on student records from SIMS, results for each administration will be printed. If a student participated in only the spring 2011 administration, or if his or her records from previous administrations could not be matched (based on SIMS), results from only the spring 2011 administration will be printed. Grade This column indicates the grade in which the student was enrolled at the time of testing, as reported by the district in the March 2011 SIMS submission. If a student who is not in SIMS participated in MEPA testing, the student’s grade was indicated on his or her answer booklet by designated school personnel. If no grade information was available for a student, a (–) is displayed. Each student’s results are included in the grade span report for the grade span test in which the student participated in spring 2011. If the spring 2011 grade displayed on the report does not match the grade span of the report in which it is included, it may indicate that student information in SIMS needs to be updated, or that information on the student’s answer booklet was bubbled incorrectly. MEPA-R/W The columns under MEPA-R/W show which session combination (Sessions 1 and 2 or Sessions 2 and 3 for students in grades 3–12) or level (Level A or Level B for students in grade span K–2) each student took for each subtest (reading and writing), as well as a scaled score for each subtest. The scaled score range for both reading and writing is 0–30. If a student did not participate in either reading or writing, a non-participation code is displayed in the scaled score column(s). Non-participation codes are listed in the table on page 13. Students who receive a non-participation code will not be counted as participating for purposes of accountability. Students who used a nonstandard accommodation have a footnote indicated on the right-hand side of their reading and/or writing scaled score. A nonstandard accommodation in reading is indicated with an a and a footnote indicating that the reading test was read aloud to the student as required by his or her Individualized Education Program (IEP) or 504 plan. A nonstandard accommodation in writing is indicated with a b and a footnote indicating that the writing test was scribed for the student as required by his or her IEP or 504 plan. MELA-O These columns show the student’s spring 2011 listening raw score (range 0–5) and speaking raw score (range 0–20). The speaking raw score is the total of four speaking subscores: fluency (0–5), vocabulary (0–5), pronunciation (0–5), and grammar (0–5). MELA-O scores were reported by designated school staff on MEPA answer booklets. If a student did not participate in the MELA-O, or if MELA-O scores were incomplete or not indicated on the student’s answer booklet, a non-participation code is displayed in the Listening Raw Score and Speaking Raw Score columns. Based on Department policy, only students who are deaf or hard of hearing are exempt from participating in the MELA-O. Overall MEPA Results The columns in this section of the report show overall student performance on the MEPA test. If a scaled score could not be calculated for the spring 2011 MEPA test administration, a (–) is displayed in the Overall Scaled Score column. The Performance Level column indicates the student’s performance level based on his or her overall scaled score. If a student did not participate in the MEPA-R/W or MELA-O, or had incomplete scores in one or more of the tested learning areas, no performance level is indicated and a non-participation code is displayed instead. Non-Participation Code Explanation MED A student did not participate in the test due to a medically documented absence. ANA A student did not participate in the test for one of the following reasons: • a required test accommodation was not available for this test; • alternate assessment was not available for this test; or • the student is considered deaf or hard of hearing. ABS A student did not participate due to an absence that was not medically documented. TRN A student did not participate because he or she transferred to or from another school during testing time and did not participate in the required test sessions. INV A student’s results are invalidated if he or she was tested but was not required to be tested, or if a testing irregularity was reported to the Department. The Made Progress column indicates whether the student made progress toward acquiring English language proficiency. Students who do not have a baseline performance level from spring 2010 or fall 2010 are reported as NA. V. Understanding the School and District Preliminary Participation Report and Preliminary Results by Year of Enrollment in Massachusetts Schools Overview A Preliminary Participation Report package has been produced by district and school for each grade span (K–2, 3–4, 5–6, 7–8, and 9–12) tested in spring 2011. Each package consists of two reports: 1. Preliminary Participation Report 2. Preliminary Results by Year of Enrollment in Massachusetts Schools School Identification The box in the upper right corner of each page (as shown below) shows the school name, its eight-digit identification number, and the district name for which the report has been generated. [Fictitious School Identification Label] Minimum Number of Students Needed to Report Results A report is generated for each school in which a minimum number of students was tested at a given grade span. In order to ensure confidentiality of individual student results and discourage generalizations about school performance based on very small populations, the Department of Elementary and Secondary Education has established a minimum number of students who must be tested in order for school results to be reported. For schools and districts that tested fewer than 10 students in any particular grade span, results are suppressed for that grade span on the report. Information Included in the Spring 2011 MEPA Preliminary Participation Report This report provides preliminary participation information for students who participated in the spring 2011 MEPA tests, for both the MEPA-R/W and MELA-O individually and for the two assessments combined. For each grade span tested in the school, the school and district receive a separate Preliminary Participation Report each that includes the following information: A. Number of Students Enrolled B. Number Tested C. Percent Tested D. Number Not Tested A. Number of Students Enrolled The Number of Students Enrolled displayed in column 2 is the total number of ELL students enrolled based on March 2011 SIMS data, regardless of their participation status. The Number of Students Enrolled equals the sum of the Number Tested and the Number Not Tested. B. Number Tested Column 3 displays the total number of students who participated in the test. C. Percent Tested Column 4 shows the percentage of enrolled students who were tested on each MEPA assessment (MEPA-R/W and MELA-O) and the two assessments combined. D. Number Not Tested Columns 5–11 show the number of students who did not participate in one or more of the MEPA tests. Absent—Medically Documented Column 5 shows the number of students who were reported as Absent—Medically Documented during the testing period in spring 2011 as indicated by designated school personnel on answer booklets. Students in this category are those who obtained a written statement from a licensed medical doctor, stating both the medical reason that prevented them from taking the test and the dates for which they were absent due to the medical condition. Number (#) of Students Reported in SIMS as LEP but Not Tested Column 6 shows the number of ELL students enrolled at the time of testing, according to March 2011 SIMS data, who were not tested. Not Tested Other Absence Column 7 shows the number of students who were reported as Not Tested Other Absence during the testing period in spring 2011 as indicated by designated school personnel on answer booklets. Reasons that students were given this non-participation code include the following: • Student was absent without medical documentation. • There were no responses to test items in the answer booklet, and no non-participation code was indicated on the answer booklet for either reading or writing. • MELA-O scores were missing from or were incomplete on the answer booklet, and the student is not identified as deaf or hard of hearing for both reading and writing. • Two separate answer booklets were submitted for the student and both answer booklets contained responses to the same items from the same session combinations for reading and/or writing. • The student’s results were invalidated because the student was not required to be tested or because a testing irregularity was reported to the Department. Required Test Accommodations Not Available Column 8 shows the number of students who did not participate in the MEPA-R/W because a required accommodation indicated in the student’s IEP was not available for the reading and/or writing tests. This determination was indicated by designated school personnel on student answer booklets. Note, all ELL students (except those who are deaf or hard of hearing) were required to participate in the MELA-O. Deaf/Hard of Hearing Column 9 shows the number of students who were reported as not tested because they are deaf or hard of hearing as indicated by designated school personnel on answer booklets. Alternate Assessment Not Available Column 10 shows the number of students who did not participate in the MEPA-R/W because an alternate assessment, as required by the student’s IEP, was not available for the reading and/or writing tests. This determination was indicated by designated school personnel on student answer booklets. Note that all ELL students (except those who are deaf or hard of hearing) were required to participate in the MELA-O. Transferred During Testing Column 11 shows the number of students who did not participate because the students transferred to or from another school during the testing window and did not participate in the required test sessions. Information Included in the Spring 2011 MEPA Preliminary Results by Year of Enrollment in Massachusetts Schools This report shows preliminary school, district, and state results for students according to the number of years they had been enrolled in Massachusetts public schools at the time of testing. The number of years a student had been enrolled at the time of the spring 2011 test administration is based on March 2011 SIMS data. The table on page 11 shows how enrollment duration is categorized. The results in this report include the following: A. Preliminary Participation Results for MEPA-R/W and MELA-O B. Average Scaled Score Results C. Performance Level Results D. Progress Results A. Preliminary Participation Results for MEPA-R/W and MELA-O The # Enrolled in column 2 is the total number of ELL students enrolled based on March 2011 SIMS data, regardless of their non-participation status. The # Tested in column 3 shows the number of students who were tested and have complete scores in all of the tested areas (reading, writing, listening, and speaking). The % Tested in column 4 shows the percentage of students who were tested and have complete scores in all of the tested areas. This number is calculated by dividing the # Tested in column 3 by the # Enrolled in column 2. B. Average Scaled Score Results The number shown in column 5 provides the average overall MEPA scaled score according to the number of years students had been enrolled in Massachusetts public schools. The scaled score range for MEPA is 400–550. Only students who were tested and have complete scores in all of the tested areas are included in the average. C. Performance Level Results Performance level results shown in columns 6–15 provide the number and percentage of students performing at each of the five MEPA performance levels (Level 1, Level 2, Level 3, Level 4, and Level 5) according to the number of years students have been enrolled in Massachusetts public schools. These numbers and percentages are based on the results of all students who were tested and who have complete scores in all of the tested areas. D. Progress Results Columns 16 and 17 indicate the number and percentage of students who made progress toward acquiring English language proficiency according to the number of years the students have been enrolled in Massachusetts public schools. These determinations are based on students who were tested and have complete scores in all of the tested areas. VI. Understanding the 2011 Progress Report Overview The 2011 Progress Report provides information at the district and school levels for each grade span (K–2, 3–4, 5–6, 7–8, and 9–12) tested in spring 2011. The progress determinations are based on ELL students who participated in all portions of the MEPA test (reading, writing, listening, and speaking) in spring 2011 and who had a performance level from spring 2010 or fall 2010 that was used as a baseline on which to determine progress. The definition of progress is based on research into typical language acquisition patterns and the amount of improvement needed each year by a beginning ELL student in order to transition out of ELL services in five to six years. Each of the lower four performance levels (Levels 1–4) is divided into two performance level steps (Low and High). The highest level (Level 5) is divided into five 10-point steps. For students whose baseline MEPA score is from the same grade-span test as in spring 2011, the Department defines progress toward acquiring English language proficiency as advancing two or more performance level steps until they reach Level 3 High. After a student reaches Level 3 High, progress is defined as advancing one performance level step. For students whose baseline MEPA score is from a lower grade-span test than the test taken in spring 2011, progress is defined as advancing at least one performance level step until they reach Level 5, at which point they are considered to be making progress if they maintain Level 5 status. Summary data are suppressed if fewer than 10 students are included. Progress of Students Tested in Spring 2011 [Progress of Students Tested in Spring 2011 chart] The first column in this chart indicates grade span, with the first row showing “All grades” and the following rows showing the individual MEPA grade spans (K–2, 3–4, 5–6, 7–8, and 9–12). Information in the remaining columns is reported by grade span, with the total being displayed next to “All grades.” If a school had fewer than 10 students in a given grade span included, the number and percentage of students making progress are not shown for that grade span. The second column indicates the number of students from the school or district who were included in each grade-span calculation. The third column indicates the number of students from the school or district who made progress since the last time they were tested and received a performance level. The fourth column is a bar chart showing the percentage of students who made progress based on the number of students included for that school or district. Students in Level 4 High and Level 5 in Spring 2011 by Number of Years in Massachusetts Public Schools [Students in Level 4 High and Level 5 in Spring 2011 by Number of years in Massachusetts Public Schools chart] Summary data are suppressed if fewer than 10 students are included. This table contains a bar graph for each grade span. The number of consecutive years students have been enrolled in Massachusetts public schools is displayed on the horizontal axis, with the percentage of students displayed on the vertical axis. At the top of each bar is the number of students from the school or district who are in Level 4 High or Level 5 based on spring 2011 results and the percentage these students represent of the total number of students in the school or district who have been in Massachusetts schools for the same number of years. If a school had no ELL students in a given grade span or if no students had been in Massachusetts schools for the number of years indicated, no bar is displayed in the corresponding section of the graph. Comparison of Performance Levels of Students Tested in Spring 2011 and Spring or Fall 2010 [Comparison of Performance Levels of Students Tested in Spring 2011 and Spring or Fall 2010] This chart shows a comparison of performance levels for students who tested in spring 2011 and spring or fall 2010. A school or district progress report may contain multiple charts in order to compare students who changed grade spans between test administrations and students who remained in the same grade span. Each table has the previous administration shown on the vertical axis and the spring 2011 results displayed on the horizontal axis. Performance levels 1–4 are each divided into two steps, Low and High. Performance level 5 is divided into five 10-point steps. The chart displays the number and percentage of students included in each performance level step. Blocks were shaded to indicate that progress was made or were left white to indicate that progress was not made. The following is a description of the number of steps each student must advance from one year to the next in order to be considered as making progress. For students who took the same grade-span test in spring 2011 and in 2010 (e.g., grade span 5–6 in both years): 2010 MEPA performance level step Needed to make progress in 2011 Level 1 Low to Level 3 Low Must advance 2 steps Level 3 High to Level 4 High Must advance 1 step Level 5 Must advance 1 step For students who took a different grade-span test in spring 2011 and in 2010 (e.g., grade span 5–6 in spring 2011 and grade span 3–4 in 2010): 2010 MEPA performance level step Needed to make progress in 2011 Level 1 Low to Level 4 High Must advance 1 step Level 5 Must maintain Level 5 Appendix MEPA OVERALL RAW SCORE TO SCALED SCORE CONVERSION TABLES Table 1. Grade Span K–2 [Table 1. Grade Span K–2] Table 1. Grade Span K–2 (continued) [Table 1. Grade Span K–2 (continued)] Table 2. Grade Span 3–4 [Table 2. Grade Span 3–4] Table 2. Grade Span 3–4 (continued) [Table 2. Grade Span 3–4 (continued)] Table 2. Grade Span 3–4 (continued) [Table 2. Grade Span 3–4 (continued)] Table 3. Grade Span 5–6 [Table 3. Grade Span 5–6] Table 3. Grade Span 5–6 (continued) [Table 3. Grade Span 5–6 (continued)] Table 3. Grade Span 5–6 (continued) [Table 3. Grade Span 5–6 (continued)] Table 4. Grade Span 7–8 [Table 4. Grade Span 7–8] Table 4. Grade Span 7–8 (continued) [Table 4. Grade Span 7–8 (continued)] Table 4. Grade Span 7–8 (continued) [Table 4. Grade Span 7–8 (continued)] Table 5. Grade Span 9–12 [Table 5. Grade Span 9–12] Table 5. Grade Span 9–12 (continued) [Table 5. Grade Span 9–12 (continued)] Table 5. Grade Span 9–12 (continued) [Table 5. Grade Span 9–12 (continued)] Table 6. Grade Span K–2 Reading Level A* [Table 6. Grade Span K–2 Reading Level A] ^*The raw score range for Level A Reading is 0–16. Table 7. Grade Span K–2 Reading Level B* [Table 7. Grade Span K–2 Reading Level B] ^*The raw score range for Level B Reading is 0–14. Table 8. Grade Span K–2 Writing Level A* [Table 8. Grade Span K–2 Writing Level A] ^*The raw score range for Level A Writing is 0–12. Table 9. Grade Span K–2 Writing Level B* [Table 9. Grade Span K–2 Writing Level B] ^*The raw score range for Level B Writing is 0–20. Table 10. Grade Span 3–4 Reading Sessions 1 & 2* [Table 10. Grade Span 3–4 Reading Sessions 1 & 2] ^*The raw score range for Reading Sessions 1 & 2 is 0–27. Table 11. Grade Span 3–4 Reading Sessions 2 & 3* [Table 11. Grade Span 3–4 Reading Sessions 2 & 3] ^*The raw score range for Reading Sessions 2 & 3 is 0–26. Table 12. Grade Span 3–4 Writing Sessions 1 & 2* [Table 12. Grade Span 3–4 Writing Sessions 1 & 2] ^*The raw score range for Writing Sessions 1 & 2 is 0–24. Table 13. Grade Span 3–4 Writing Sessions 2 & 3* [Table 13. Grade Span 3–4 Writing Sessions 2 & 3] ^*The raw score range for Writing Sessions 2 & 3 is 0–26. Table 14. Grade Span 5–6 Reading Sessions 1 & 2* [Table 14. Grade Span 5–6 Reading Sessions 1 & 2] ^*The raw score range for Reading Sessions 1 & 2 is 0–27. Table 15. Grade Span 5–6 Reading Sessions 2 & 3* [Table 15. Grade Span 5–6 Reading Sessions 2 & 3] ^*The raw score range for Reading Sessions 2 & 3 is 0–26. Table 16. Grade Span 5–6 Writing Sessions 1 & 2* [Table 16. Grade Span 5–6 Writing Sessions 1 & 2] ^*The raw score range for Writing Sessions 1 & 2 is 0–24. Table 17. Grade Span 5–6 Writing Sessions 2 & 3* [Table 17. Grade Span 5–6 Writing Sessions 2 & 3] ^*The raw score range for Writing Sessions 2 & 3 is 0–26. Table 18. Grade Span 7–8 Reading Sessions 1 & 2* [Table 18. Grade Span 7–8 Reading Sessions 1 & 2] ^*The raw score range for Reading Sessions 1 & 2 is 0–27. Table 19. Grade Span 7–8 Reading Sessions 2 & 3* [Table 19. Grade Span 7–8 Reading Sessions 2 & 3] ^*The raw score range for Reading Sessions 2 & 3 is 0–26. Table 20. Grade Span 7–8 Writing Sessions 1 & 2* [Table 20. Grade Span 7–8 Writing Sessions 1 & 2] ^*The raw score range for Writing Sessions 1 & 2 is 0–24. Table 21. Grade Span 7–8 Writing Sessions 2 & 3* [Table 21. Grade Span 7–8 Writing Sessions 2 & 3] ^*The raw score range for Writing Sessions 2 & 3 is 0–26. Table 22. Grade Span 9–12 Reading Sessions 1 & 2* [Table 22. Grade Span 9–12 Reading Sessions 1 & 2] ^*The raw score range for Reading Sessions 1 & 2 is 0–27. Table 23. Grade Span 9–12 Reading Sessions 2 & 3* [Table 23. Grade Span 9–12 Reading Sessions 2 & 3] ^*The raw score range for Reading Sessions 2 & 3 is 0–26. Table 24. Grade Span 9–12 Level A Writing Sessions 1 & 2* [Table 24. Grade Span 9–12 Level A Writing Sessions 1 & 2] ^*The raw score range for Writing Sessions 1 & 2 is 0–24. Table 25. Grade Span 9–12 Level B Writing Sessions 2 & 3* [Table 25. Grade Span 9–12 Level B Writing Sessions 2 & 3] ^*The raw score range for Writing Sessions 2 & 3 is 0–26. Page 16 of 17 Page 15 of 17