[MCAS Logo] Guide to Interpreting the Spring 2008 MCAS Reports for Schools and Districts September 2008 Massachusetts Department of Elementary and Secondary Education [Massachusetts DESE logo] This document was prepared by the Massachusetts Department of Elementary and Secondary Education Mitchell D. Chester, Ed.D. Commissioner of Elementary and Secondary Education © 2008 Massachusetts Department of Elementary and Secondary Education Permission is hereby granted to copy for non-commercial educational purposes any or all parts of this document. Please credit the “Massachusetts Department of Elementary and Secondary Education.” Massachusetts Department of Elementary and Secondary Education 350 Main Street, Malden, MA 02148-5023 Phone 781-338-3000 TTY: N.E.T. Relay 800-439-2370 www.doe.mass.edu [State Seal] Table of Contents I. Document Purpose.................................................................................................1 II. Reporting of the 2008 MCAS Test Results..............................................................2 III. General Guidelines for Interpretation and Use of MCAS Reports............................3 Use of MCAS Results.............................................................................................3 Comparing Scores from Year to Year and Test to Test.............................................3 Common and Matrix-Sampled Item Results...........................................................3 Composite Performance Index (CPI) Points.............................................................4 Conversion of Raw Scores to Scaled Scores.............................................................5 Comparisons of Results.........................................................................................5 References to LEP Students.....................................................................................6 State Regulations Regarding Confidentiality of Student Records.............................6 IV. History of MCAS Standard Setting........................................................................7 V. Understanding the Test Item Analysis Roster and the Test Item Analysis Report Summary.................................................................8 Overview...............................................................................................................8 Test Item Analysis Roster.....................................................................................10 Test Item Analysis Report Summary.....................................................................17 VI. Understanding the School Report and District Report...........................................21 Overview.............................................................................................................21 A. Student Participation Report..........................................................................23 B. Distribution of Results by Composite Performance Index Points......................26 C. Subject Area Subscores on Standard Test.........................................................28 D. Performance Level Results..............................................................................30 E. Three-Year Comparison of School Performance Level Results.........................31 F. Performance Level and CPI for Subgroups of Students.....................................33 G. Competency Determination Results (Grade 10 Only)......................................34 VII. Understanding the 2008 High School Science and Technology/Engineering Participation and Performance Reports...........................................................35 Appendix A: Raw Score to Scaled Score Conversion Tables.........................................36 Appendix B: Calculating Threshold Scores (Cut Points)..............................................50 Appendix C: MCAS-Related References.....................................................................52 I. Document Purpose The purpose of this document is to support local educators’ use of district and school reports of results from the spring 2008 administration of the Massachusetts Comprehensive Assessment System (MCAS) tests. This document provides general information about MCAS results, standards, and reports, with specific guidance on interpreting the following MCAS reports for the spring 2008 tests: • Test Item Analysis Roster • Test Item Analysis Report Summary • School Report • District Report In their efforts to better serve the academic needs of all students, local educators can use the important information in these 2008 MCAS reports to improve curriculum and instruction. In addition, this document can help school and district personnel communicate with parents and members of their communities about MCAS test results. II. Reporting of the 2008 MCAS Test Results The table below shows the MCAS tests and pilot tests administered in spring 2008, and how each test’s results are reported. TABLE 1. MCAS RESULTS REPORTED IN 2008 Grade Subject Area Raw Scaled Performance Test Item Score Score Level Analysis 3 English Language Arts (ELA) Reading Comprehension^1 * * * Mathematics^1 * * * 4 ELA Composition and Reading Comprehension * * * * Mathematics * * * * 5 ELA Reading Comprehension * * * * Mathematics * * * * Science and Technology/Engineering * * * * History and Social Science Pilot Test^2 * * 6 ELA Reading Comprehension * * * * Mathematics * * * * 7 ELA Composition and Reading Comprehension * * * * Mathematics * * * * History and Social Science Pilot Test^2 * * 8 ELA Reading Comprehension * * * * Mathematics * * * * Science and Technology/Engineering * * * * High High School Science and * * * * School Technology/Engineering • Biology • Chemistry • Introductory Physics • Technology/Engineering 10 ELA Composition and Reading Comprehension * * * * Mathematics * * * * High U.S. History Pilot Test^2 * * School * = data reported [box] = data not reported NOTES to TABLE 1: ^1. Results from the MCAS grade 3 ELA Reading Comprehension and Mathematics tests are reported by performance level and raw score; no scaled scores are reported for grade 3 tests. ^2. A pilot administration of History and Social Science tests for grades 5 and 7 and a high school U.S. History test took place in spring 2008. Only Test Item Analysis Rosters and Test Item Analysis Summary Reports are issued for these pilot tests; these reports supply valuable information to schools and districts regarding the alignment of curricula to the History and Social Science Curriculum Framework learning standards. III. General Guidelines for Interpretation and Use of MCAS Reports USE OF MCAS RESULTS Results contained in the 2008 MCAS reports illustrate the extent to which students and schools are achieving the learning standards contained in the Massachusetts Curriculum Frameworks. The Frameworks are available on the Department of Elementary and Secondary Education’s (“the Department”) website at www.doe.mass.edu/frameworks/current.html. Improvement of Student Performance Teachers and parents can use individual student results and school-level results to focus on areas in which students need assistance to improve performance. Alignment of Curriculum and Instruction Teachers and administrators can use school- and district-level results to evaluate the effectiveness of local curriculum and instruction in aligning with the standards defined in the Frameworks. COMPARING SCORES FROM YEAR TO YEAR AND TEST TO TEST Because MCAS tests are different each year, it is necessary to link the tests through a process called equating. Equating is used to determine the cutscores on each subsequent MCAS test that will maintain the performance standards established on the original test. If the original and subsequent tests were identical in difficulty, the cutscores would be the same on each test. However, it is very challenging to develop two tests that are exactly equal in difficulty. Each subsequent test is likely to be slightly easier or slightly more difficult than the original test. If a subsequent test is easier, the cutscores needed to move from one performance level to the next will be slightly higher. If a subsequent test is more difficult than the original, the cutscores needed to move from one performance level to the next will be slightly lower. For the current MCAS program, tests are equated each year using an anchor test that is embedded within the actual MCAS test. The items on the anchor test do not count toward a student’s score on the test. The items are only used to link performance standards on the original and subsequent MCAS tests. Items on the anchor test are not released to the public after each test administration so that they can be used again to link future tests. The anchor test measures the same content and skills measured by the actual MCAS test. Because of the critical importance of maintaining the same performance standards across years, the Department takes extra precautions to ensure the accuracy of the equating process. Each year, psychometricians from the Department’s assessment contractor and the Center for Educational Assessment at UMASS Amherst independently and simultaneously equate the MCAS tests. The Department, with assistance as needed from its national Technical Advisory Committee, analyzes the results of the two independent equating analyses prior to reporting any MCAS test results. COMMON AND MATRIX-SAMPLED ITEM RESULTS All MCAS student-level scaled score (raw score at grade 3) and performance level results are based on common items only. The Test Item Analysis Roster and Test Item Analysis Report Summary report results for only common items. In addition to scaled score (raw score at grade 3) and performance level results based on common items, the School Report and District Report contain aggregated subscores for specific content area strands. These school- and district-level subscores include student responses to both common and matrix-sampled MCAS items. COMPOSITE PERFORMANCE INDEX (CPI) POINTS Composite Performance Index (CPI) points are awarded to schools and districts for each student tested. October 2007 enrollment data were used to determine school CPI points; spring 2008 enrollment data were used to determine district and state CPI points. CPI points are assigned for each student according to the tables below. Standard MCAS Tests CPI points for students who took the standard MCAS tests are awarded to schools and districts based on performance level categories and scaled scores (raw scores at grade 3). TABLE 2. ASSIGNMENT OF CPI POINTS FOR STANDARD MCAS TESTS GRADE 3 MCAS Performance Level Reading Mathematics CPI Points per Student Comprehension Raw Score Raw Score Above Proficient 43–48 36–40 100 Proficient 36–42 30–35 100 Needs Improvement—High 29–35 26–29 75 Needs Improvement—Low 23–28 22–25 50 Warning—High 11–22 8–21 25 Warning—Low 0–10 0–7 0 TABLE 3. ASSIGNMENT OF CPI POINTS FOR STANDARD MCAS TESTS GRADES 4–8 AND HIGH SCHOOL MCAS Performance Level Scaled Score CPI Points per Student Advanced 260–280 100 Proficient 240–258 100 Needs Improvement—High 230–238 75 Needs Improvement—Low 220–228 50 Warning/Failing*—High 210–218 25 Warning/Failing*—Low 200–208 0 *The MCAS performance level of Failing is used only for high school test results. Alternate Assessments Since no scaled scores are reported for students taking the Alternate Assessment, CPI points for these students are awarded based on Alternate Assessment performance level categories only. TABLE 4. ASSIGNMENT OF CPI POINTS FOR ALTERNATE ASSESSMENT GRADE 3 Alternate Assessment CPI Points Standard MCAS Performance Level per Student Performance Level Above Proficient (P+A) 100 Above Proficient Proficient (P_A) 100 Proficient Needs Improvement (NIA) 100 Needs Improvement Progressing (PRG) 100 Warning Emerging (EMG) 75 Warning Awareness (AWR) 50 Warning Incomplete Portfolio (INP) 25 Warning TABLE 5. ASSIGNMENT OF CPI POINTS FOR ALTERNATE ASSESSMENT GRADES 4–8 AND HIGH SCHOOL Alternate Assessment CPI Points Standard MCAS Performance Level per Student Performance Level Advanced (A_A) 100 Advanced Proficient (P_A) 100 Proficient Needs Improvement (NIA) 100 Needs Improvement Progressing (PRG) 100 Warning/Failing* Emerging (EMG) 75 Warning/Failing* Awareness (AWR) 50 Warning/Failing* Incomplete Portfolio (INP) 25 Warning/Failing* *The standard MCAS performance level of Failing is used only for high school test results. CONVERSION OF RAW SCORES TO SCALED SCORES Scaled scores are derived from raw scores (based on common items only) using a data analysis process known as scaling; the scaling process is analogous to converting temperature from degrees Fahrenheit to degrees Celsius. Conversion of raw scores to scaled scores is done each year to ensure that the standard for landing in each performance level remains true to the standards set by the original committee. Appendix A contains the 2008 MCAS Raw Score to Scaled Score Conversion Tables for each test administered in spring 2008. Appendix B contains an explanation of how threshold scores for each performance level are determined. Further information about MCAS scaling methods can be found in the MCAS 2007 Technical Report, which can be accessed by going to www.mcasservicecenter.com and clicking “MCAS.” COMPARISONS OF RESULTS Performance levels are the most appropriate measure to use when comparing MCAS results among students, schools, and districts. Results from the 2008 MCAS tests should be evaluated in conjunction with results from MCAS tests administered from 1998 through 2007. As with any evaluation, school and district test results are most meaningful when compared with other indicators and when examined over several years for long-term trends in performance. More information regarding score interpretations over time is available on page 3 (“Comparing Scores from Year to Year and Test to Test”). REFERENCES TO LEP STUDENTS Limited English Proficient (LEP) students have been designated as LEP in the Student Information Management System (SIMS). In reports for schools and districts, the code of LEP refers to either first-year LEP students or to all LEP students except those in their first year. The code’s meaning depends on its context: 1. Results Reporting When reporting results, the code LEP refers to those limited English proficient students who participated in MCAS testing and who are not first-year LEP students. (The code FLEP [Formerly Limited English Proficient] refers to those students who are not currently classified as LEP but who were classified as LEP at any time within the past three years.) 2. Performance Level Coding When the code LEP appears as a performance level designation, it refers to a limited English proficient student in his or her first year of enrollment in U.S. schools. First-year LEP students who participated in the Massachusetts English Proficiency Assessment (MEPA) are not ordinarily included in results other than AYP participation rates. The ESE uses SIMS enrollment records to confirm first-year status. STATE REGULATIONS REGARDING CONFIDENTIALITY OF STUDENT RECORDS The Massachusetts Student Record Regulations (603 CMR 23.00) and the federal Family Educational Rights and Privacy Act (FERPA) restrict access to individual student results, including those provided in MCAS Test Item Analysis Rosters and Parent/Guardian Reports. A student’s temporary records contain information that does not appear in the student’s transcript. These temporary records generally cannot be examined by anyone other than the student, his or her parents or guardians, and authorized school personnel, unless the parent/guardian or student gives consent. A student’s temporary records are maintained for at least five years after he or she transfers, graduates, or withdraws from the school system. Superintendents and principals are responsible for ensuring the privacy and security of all student records. In accordance with state and federal regulations, authorized school personnel shall have access to the records of students to whom they are providing services only when such access is required for the performance of their official duties. For more information on this topic, refer to the Massachusetts Department of Elementary and Secondary Education publication, Student Records: Questions, Answers, and Guidelines. IV. History of MCAS Standard Setting The MCAS tests were first administered to students in Massachusetts in 1998. At that time, English Language Arts (ELA) and Mathematics were the subjects assessed at grades 4, 8, and 10. The No Child Left Behind law (NCLB) required that students be tested in grades 3 through 8 and 10 by 2006. In order to meet these requirements, it was necessary to add tests for the grades and subjects that were not previously included. From 2001 to 2006, additional grades and subject areas were tested. Following the initial administration of each new test, performance standards were set. NCLB also required that four performance levels be reported. Therefore, in 2006, a fourth performance level (Above Proficient) was established for grade 3 Reading. A meaningful third performance level cut point was reset between Proficient and Above Proficient for grade 3 Mathematics in summer 2007. The high school Science and Technology/Engineering (STE) subjects were operationally assessed for the first time in 2007. Table 6 displays the years (1998–2007) in which standards were set for various subjects by grade, and the standard- setting method used. TABLE 6. STANDARD-SETTING HISTORY 1998–2007 Grade Content Year Method 3 Reading 2001/2006 Bookmark/Body of Work 3 Mathematics 2006/2007 Body of Work 4 ELA 1998/2001 Body of Work 4 Mathematics 1998 Body of Work 5 Language & Literature 2006 Body of Work 5 Mathematics 2006 Body of Work 5 Science and Technology/Engineering 2003 Body of Work 6 Language & Literature 2006 Body of Work 6 Mathematics 2001 Body of Work 7 ELA 2001 Body of Work 7 Mathematics 2006 Body of Work 8 ELA 1998/2006 Body of Work 8 Mathematics 1998 Body of Work 8 Science and Technology/Engineering 2003 Body of Work 10 ELA 1998 Body of Work 10 Mathematics 1998 Body of Work High School Science and Technology/Engineering 2007 Body of Work V. Understanding the Test Item Analysis Roster and the Test Item Analysis Report Summary OVERVIEW The Test Item Analysis Roster and Test Item Analysis Report Summary provide results from all MCAS tests administered in spring 2008, including pilot tests. The information provided in these reports can be used to • identify strengths and weaknesses in subject area knowledge and skills exhibited by students • evaluate and plan curricula in the classroom, school, and district • summarize and discuss results with parents and the public (see “State Regulations Regarding Confidentiality of Student Records” on page 6) The Test Item Analysis Roster Test Item Analysis Rosters show student-level results for individual MCAS common items. Each school receives a separate Test Item Analysis Roster for each subject area and grade tested in the school. For each test, the Roster shows how each student • answered each common multiple-choice question • performed on each common open-response question • performed on each short-answer question, where applicable • performed on the English Language Arts Composition (writing prompt), where applicable The Roster is designed to be used in conjunction with the publication Release of Spring 2008 MCAS Test Items, which can be found on the Department website at www.doe.mass.edu/mcas/testitems.html. The Release of Spring 2008 MCAS Test Items shows all the questions on which Test Item Analysis Rosters are based, providing educators with a detailed picture of each student’s MCAS performance. This picture should be used in combination with the student’s classroom performance to develop an overview of his or her academic achievement. The Test Item Analysis Report Summary The Test Item Analysis Report Summary shows overall performance at the school or district level, and at the state level, for individual common items. Each school and district receives a separate Test Item Analysis Report Summary for each subject area and grade tested in the school or district. Rosters and Summaries for Specific Tests Grades 4, 7, and 10 English Language Arts (ELA) Rosters and Summaries differ slightly from these reports for other tests, since they include results for the ELA Composition, as described on the following pages. There are four high school Science and Technology/Engineering (STE) test item analysis rosters and test item analysis report summaries, one for each of the four STE subjects: Biology, Chemistry, Introductory Physics, and Technology/Engineering. These rosters and summaries combine results from students in grades 9 and 10. Specifically, the rosters include grade 9 students who participated in an STE test in 2008 and all grade 10 students (based on SIMS). The summaries include grade 9 and grade 10 students who participated in STE in 2008. In addition to these rosters and summaries, the Roster of Class of 2010 Students’ Performance Levels through Spring 2008 is also provided. This roster summarizes ELA, Mathematics, and STE results for all members of the class of 2010 (as determined by June 2008 SIMS). The information on the roster is based on the highest MCAS results each student has earned to date in high school. The columns at the far right of the roster indicate whether a student is included in summary performance results for each subject at both the school and district levels; inclusion is based on SIMS records. The policies for inclusion are outlined on the legend page of the roster. ELA and Mathematics retest results are reported only at the student level (in Test Item Analysis Reports); retest results are not aggregated at school or district levels. Test Item Analysis Roster [Sample report of Test Item Analysis Roster] SAMPLE REPORT- FICTITIOUS DATA TEST ITEM ANALYSIS ROSTER Each Test Item Analysis Roster has three main sections: 1. Item Information 2. Individual Student Results 3. Legend The number of pages in the Roster depends on the total number of students in the tested grade. 1. Item Information [Screen shot sample of Item Information depicting Item number, Item type, Correct MC answer, and Reporting category] Section 1 of the Roster (as indicated on page 10) contains four types of information for each common item, listed vertically: • item number • item type • correct multiple-choice (MC) answer, if applicable • reporting category Section 1 also shows the total raw score points possible for the reported test. Item Number Item Number corresponds to the number of the question as it appears in the Release of Spring 2007 MCAS Test Items publication. For grades 4, 7, and 10 ELA only, this information is blank for the first two columns, which provide ELA Composition data. Item Type Item Type indicates whether the item was • multiple-choice (MC) • open-response (OR) • (for Mathematics only) short-answer (SA) • (for grades 4, 7, and 10 ELA only) ELA Composition writing prompt (“WP” appears in the first two columns) See page 12 (“Individual Student Results”) for further information about interpreting Item Type data. Correct MC Answer Correct MC Answer provides the letter option (A, B, C, or D) for the correct answer for multiple-choice questions. Reporting Category Each Reporting Category is a strand of the Curriculum Framework for the tested subject area (e.g., Reading and Literature is a strand of the English Language Arts Framework and an MCAS reporting category). For grades 4, 7, and 10 ELA, only the first two columns relate to performance on the composition. Reporting category codes are explained in the Legend at the end of the Test Item Analysis Roster (see page 16). Total Possible Points The number shown in the fourth-from-last column for Total Poss. Points is the sum of the maximum raw score points for all common questions on the test: 1 point for each MC question + 4 points for each OR question (2 points for grade 3 Mathematics OR questions) + 1 point for each SA question (Mathematics only) + 20 points for each WP (grades 4, 7, and 10 ELA only: includes points for both WP columns) 2. Individual Student Results [Screen shot sample of Body Roster depicting Individual Student Results] Individual student results for each common test item are provided in section 2 (as indicated on page 16) of the Roster. Results are displayed in a separate row for each student enrolled in the school or district on the first day of testing. Results for each question are provided using various symbols, letters, and numbers, based on Item Type, as further explained below. For students who took the Alternate Assessment, all items are automatically left blank. For all other students, regardless of question type, a blank space indicates the student did not answer the question and a raw score of 0 was assigned for that question. Each student’s summary scores are provided in the final four columns of his or her row. Multiple-Choice Each multiple-choice question received a score of 0 for an incorrect answer or 1 for a correct answer. The following symbols and letters are used to indicate MC question results: • a + sign indicates the student answered the question correctly • a letter (A, B, C, or D) indicates which incorrect option the student selected • an asterisk (*) indicates the student selected more than one option and received a score of 0 on the question Open-Response Each open-response question received a score of 0–4 (except for grade 3 Mathematics open-response questions, each of which received a score of 0–2). Short-Answer Short-answer questions appeared on Mathematics tests only. Each short-answer question received a score of 0 for an incorrect answer or 1 for a correct answer. Writing Prompt Only grades 4, 7, and 10 English Language Arts Test Item Analysis Rosters show scores for ELA Compositions in the first two writing prompt (WP) columns. The first column shows the student’s score for topic development (e.g., writing development, organization, use of detail, variety in sentence structure and language). The second column shows the student’s score for standard English writing conventions (e.g., grammar, punctuation, mechanics of writing). Each student’s composition was scored independently by two scorers. Each scorer gave the composition two separate scores: • a score of 1–6 for topic development (CT) • a score of 1–4 for the student’s use of standard English writing conventions (CC) The student’s score as shown in each of these two columns (CT and CC) represents the sum of the points awarded by both scorers in each area (up to 12 points for CT; up to 8 points for CC). The Legend on the last page of the Roster displays the characteristics of student writing for each score for topic development and standard English writing conventions. Any student who wrote a totally illegible composition, wrote an off-topic response, or wrote in a language other than English was given a notation of NS (not scorable) and received a score of 0. Total Raw Score The student’s total raw score was calculated by adding one point for each MC question to the points awarded for OR and (if applicable) SA questions, and (if applicable) to writing prompt CT and CC points. Scaled Score The student’s total raw score is translated to a scaled score, reported on a scale ranging from 200–280. No scaled scores are reported for grade 3 students; this column is left blank on grade 3 Rosters. Composite Performance Index (CPI) Points (Comp. Perf. Ind. Pts.) The number in this column for each row indicates the number of CPI points awarded to the school or district based on that student’s results. Chapter III of this Guide provides further information about the award of CPI points. Performance (Perf.) Level The student’s performance level is reported as a code; performance level codes are listed and defined in the Legend of the Roster, and are also explained on the next page. Standard Performance Levels Standard Performance Level Codes Grade 3 Grades 4–8 High School P+ = Above Proficient A = Advanced A = Advanced P = Proficient P = Proficient P = Proficient NI = Needs Improvement NI = Needs Improvement NI = Needs Improvement W = Warning W = Warning F = Failing Alternate Assessment Performance Levels Seven performance levels are reported for students who participated in the MCAS Alternate Assessment, as shown below. The top three levels are equivalent to the standard MCAS performance levels with the same names. The next three (PRG, EMG, and AWR) fall within the standard MCAS Warning (Failing for grade 10) performance level. The final alternate assessment performance level, INP, corresponds to a standard MCAS “not tested” code (INC, listed under “Exceptions” on page 15). Alternate Assessment Codes A_A (P+A for grade 3) = Advanced (Above Proficient for grade 3) P_A = Proficient NIA = Needs Improvement PRG = Progressing EMG = Emerging AWR = Awareness INP = Incomplete Portfolio Certificate of Mastery Performance Levels For students who were enrolled in grade 11 or 12 and who participated in the grade 10 ELA or Mathematics test to attempt to qualify for a scholarship, the following Certificate of Mastery Codes appear in the Performance Level column. Certificate of Mastery results are excluded from school and district summaries. Certificate of Mastery Codes A_M = Advanced P_M = Proficient NIM = Needs Improvement F_M = Failing Exceptions There are limited circumstances in which scaled score and performance level results are not provided for a student; instead, in the Performance Level column, one of the following three-letter codes appears. These codes are also listed and defined in the Legend on the last page of the Roster. For further information regarding these codes, see pages 24–25 of this Guide. Not Tested Codes LEP = First-year limited English proficient TRN = Transferred ABS = Absent INC = Incomplete or invalid test results MED = Medically documented absence VAB = Voided answer booklet (appears in Preliminary Roster reports only) DUP = Multiple answer booklets submitted for this student (appears in Preliminary Roster reports only) PRE = Repeating grade 10 students who previously participated in grade 10 tests 3. Legend The Legend on the last page of each Test Item Analysis Roster provides a quick reference for interpreting the codes that appear throughout the Roster. The graphic on the following page shows the Legend from a grade 8 Mathematics Test Item Analysis Roster. For grades 4, 7, and 10 English Language Arts (ELA) Test Item Analysis Rosters only, the bottom section of the Legend also shows characteristics of student writing corresponding to each ELA Composition topic development (CT) and standard English writing conventions (CC) score. [Massachusetts Department of Elementary and Secondary Education Spring 2008 MCAS Tests Final Test Item Analysis Roster: Legend Mathematics] [Screen shot sample report of Test Item Analysis Report Summary Massachusetts Department of Elementary and Secondary Education SPRING 2008 MCAS TESTS School Test Item Analysis Report Summary Mathematics] TEST ITEM ANALYSIS REPORT SUMMARY The Test Item Analysis Report Summary for each test (by grade and subject area) shows student results, aggregated at the school and/or district level(s), and at the state level, for each common test item. Each row of the Summary provides data for one common item. Each Summary has four main sections, as indicated on pages 18–20: 1. Item Information 2. Average Item Score 3. Percent of Total Student Responses 4. Legend 1. Item Information [Screen shot of Item Information in Test Item Analysis Report Summary] The first three columns of the Summary show the Item Number, Item Type, and Reporting Category of the common item. This information correlates directly to the information provided on the Test Item Analysis Roster for the same grade and subject area (see pages 11 and 12 of this Guide for more details about these three categories). The fourth column of section 1 lists the Framework learning standard tested by that item. 2. Average Item Score [Screen shot of average item score] The Average Item Score section of each Summary shows aggregated results for each item. School Summaries include aggregated school, district, and state results for each item; district Summaries include aggregated district and state results for each item. These statistics allow educators to compare the performance of students in a school to students in the district and across Massachusetts, and to identify strengths and weaknesses in local curriculum and instruction. For multiple-choice and (on Mathematics Summaries only) short-answer questions, which are both scored 0 or 1, the aggregate score shown is equivalent to the percentage of students who answered the question correctly. For open-response questions (scored 0–4, or 0–2 in grade 3 Mathematics) and writing prompts (scored 2–20), the average student score for the question is shown. The total at the bottom of each column is the sum of the average individual item scores for the entire test (excluding the ELA writing prompt for grades 4, 7, and 10 ELA Summaries). 3. Percent of Total Student Responses [Screen shot of Percent of Total Student Responses] The six columns in section 3 (as indicated on page 17) show correct answers for multiple-choice questions (in the far right column), as well as two kinds of aggregated results: • For each multiple-choice question, the columns show the percentage of students who chose each answer option (blank, A, B, C, or D). By looking at items on which students performed well, educators can identify areas of strength in their curriculum and instruction. By focusing on items on which students made similar mistakes, educators can identify possible misconceptions held by students in the school, and can plan to address the issues by strengthening curriculum and instruction. • For open-response and (on Mathematics Summaries only) short-answer questions, the columns show the percentage of students at each score point (blank/0, 1, 2, 3, or 4 for open-response questions; blank/0 or 1 for short-answer questions). These data can be used in conjunction with item rubrics and sample student work (available on the Department website at www.doe.mass.edu/mcas/student) to inform changes in curriculum and improve instruction. 4. Legend [Test Item Analysis Summary: Legend Mathematics] The Legend on the last page of each Test Item Analysis Report Summary provides a quick reference for interpreting the codes that appear throughout the Summary. VI. Understanding the School Report and District Report OVERVIEW Each School Report and District Report provides aggregated MCAS results for the district and state; each School Report also provides aggregated results for that school. A separate Report was produced for each grade level (3–8 and 10) tested in spring 2008 and for each of the four high school Science and Technology/Engineering tests.^1 The guidelines given in this chapter may be used to interpret both the School Report and the District Report, which are nearly identical in format. All examples shown throughout this chapter refer to the School Report template. Organization of the School Report The first few pages of each School Report provide general information and definitions to help interpret the report, including a table of contents, performance level definitions, score ranges, student status definitions that relate to subgroup statistics (see section F below), and explanations regarding students who were not tested. Also provided are charts that define how Composite Performance Index (CPI) points are awarded for each test included in the Report. These introductory pages are followed by reports of spring 2008 MCAS test results. These reports include: A. Student Participation Report B. Distribution of Results by Composite Performance Index Points C. Subject Area Subscores on Standard Test D. Performance Level Results E. Three-Year Comparison of School Performance Level Results (only spring 2007 and 2008 results are available for high school Science and Technology/Engineering comparisons) F. Performance Level and Composite Performance Index for Subgroups of Students G. Competency Determination Results (grade 10 only) Each of these reports is described in further detail on the following pages. ^1 Results for grades 5 and 7 History and Social Science and high school U.S. History pilot tests are not included in School Reports or District Reports in 2008; they are included only in Test Item Analysis Rosters and Test Item Analysis Summary Reports. Further, ELA and Mathematics Retest results are not aggregated at school or district levels and do not appear in School Reports or District Reports; retest results are reported only at the student level (in Test Item Analysis Rosters). Identification [Screen shot of identification] A box in the upper right corner of each page of the School Report shows the school name, its eight-digit identification number, the district name, and the grade level for which the report has been generated. Minimum Number of Students Needed to Generate Reports In order to ensure confidentiality of individual student results and discourage generalizations about school performance based on very small populations, the Department requires that a minimum of ten students per reporting area must be tested to report school-level results in any of the following areas: • overall school performance level and scaled score results • school results disaggregated by subgroup • subject area subscore results^2 Consequently, schools with a very small number of students enrolled in a tested grade may not show results in some sections of their School Report. If a school tested fewer than ten students in a particular grade, no School Report is generated for that grade; however, results for these students are included in district- and state-level results reported in District Reports and the State Report. ^2 Multiple forms are created for each MCAS test. Matrix-sampled items are distributed across all forms. A school must administer all forms of the test for accurate computation of subject area subscores. If a school did not administer all forms of a test, the subject area subscore pages are left blank in the school’s Report. A. Student Participation Report [Screen shot of student participation report] The pages titled Student Participation Report show—in separate school, district, and state tables—how many enrolled students were included or not included in performance level results. Each table contains eight rows that report data based on student subgroup. The report has five basic sections: 1. Student Subgroups 2. Number of Students Enrolled 3. Adequate Yearly Progress Participation Rate (AYP Part. Rate) 4. Included in Performance Level Results 5. Not Included in Performance Level Results Each section is described in more detail below. 1. Student Subgroups Section 1 lists the subgroups into which data for the school’s total student population are subdivided. The total population is divided into three Student Status subgroups; it is separately divided into seven Racial/Ethnic subgroups. The three Student Status subgroups are defined as follows: • Students with Disabilities (Sw/D). Students with Disabilities are defined here as students who have an Individualized Education Program (IEP), as reported in SIMS. • Limited or Formerly Limited English Proficient (LEP/FLEP). The LEP/FLEP subgroup combines students who were LEP at the time of testing with those who have been identified as LEP in any of the past three school years. • Low-Income. A student in this subgroup meets ANY ONE of the following SIMS definitions of low- income: - The student’s family is eligible for free or reduced price lunch. - The student’s family receives Transitional Aid to Needy Families benefits. - The student’s family is eligible for food stamps. 2. Number of Students Enrolled Section 2 shows, for each student subgroup, the Number of Students Enrolled in the tested grade during the testing window. The total number of students enrolled includes both those Included in Performance Level Results (section 4) and those Not Included in Performance Level Results (section 5). 3. Adequate Yearly Progress Participation Rate (AYP Part. Rate) Section 3 shows the AYP Participation Rate for each subgroup. The number that appears on the Total row is the number of students tested in spring 2008 (on either the standard MCAS or on the MCAS Alternate Assessment) divided by the number of students enrolled on the dates of test administration. First-year LEP students who participated in the MEPA are included in the calculation, as participants (for the ELA test only). All students who have an ABS or MED test status are included in the calculations, as non-participants. Students with an INC test status are not included in the calculations. At the school level, outplaced students are not included either, unless the school was the only one in its district. 4. Included in Performance Level Results Section 4 divides subgroup results into students who were tested on the standard MCAS tests (Tested Standard) and those who participated in the Alternate Assessment (Tested Alternate Assessment). The numbers of Students with Disabilities (Sw/D) for Tested Standard include students who participated in the standard test with or without accommodations. At grade 10 only, the numbers for Limited English Proficient or Formerly Limited English Proficient (LEP/FLEP) students include those students who took the Spanish-language version of the grade 10 Mathematics test as well as those who took the English-language version of that test. (A Spanish-language version was not available for any other MCAS test.) Students who were reported as first-year LEP students on MCAS answer booklets were not ordinarily included in results other than AYP participation rate results (see section 5 below). Exceptions occurred when SIMS enrollment records indicated students had attended schools in Massachusetts before March 1, 2007. In this case, the students were not in fact first-year LEP and were therefore counted as tested; they received scaled scores and were included in School Report performance level results. 5. Not Included in Performance Level Results Section 5 shows the number of students in each subgroup who did not participate in the spring 2008 MCAS tests. The seven School Report categories for students who were Not Included in Performance Level Results are defined below. • Absent (ABS). Any student who was absent during the testing period, and who did not receive a medically documented absence as described below, received a performance level of ABS. These results are not included in the determination of scaled score, performance level, or CPI results, but are counted against the AYP participation rate. • Medically Documented Absence (MED). Any student who obtained a written statement from a licensed medical professional, stating both the medical reason that prevented him or her from taking the test and the date(s) for which he or she was absent due to the medical condition, received a performance level of MED. These results are not included in the determination of scaled score, performance level, or CPI results, but are counted against the AYP participation rate. • Previously Participated (PRE). Repeating grade 10 students who previously participated in grade 10 ELA and/or Mathematics tests receive a performance level of PRE. • Incomplete Test Results (INC). Any student whose test results were incomplete or invalid received a performance level of INC. These results are not included in the determination of scaled score, performance level, CPI, or AYP participation rate results. If, for instance, a grade 3 student takes the Mathematics test but no scorable ELA answer booklet was returned, the student’s ELA results will appear as INC. However, that student’s Mathematics results will be reported as usual and included in school- and district-level results. • First-Year LEP (LEP). LEP students in their first year of enrollment in U.S. schools receive a performance level of LEP. First-year LEP results are not included in scaled score, performance level, or CPI results, but are included in AYP participation rate results, provided that the students participated as required in MEPA. (See section 4 above for the exception to this policy.) • Transferred (TRN). Any student who transferred into or out of a school during the testing window and who did not complete all of the test sessions at that school received a performance level of TRN. These results are not included in the determination of scaled score, performance level, CPI, or AYP participation rate results. • Tested and enrolled after October 1st. Any student who enrolled in a school after October 1, 2007, was required to participate in MCAS testing but was not included in most school-level MCAS or accountability reports because the student was not enrolled for a full academic year. However, these students are included in school AYP participation rates and in district-level summaries. Further, results for these students are also included in school-level results for districts with only one school, such as charter and regional vocational districts. B. Distribution of Results by Composite Performance Index Points [Screen shot of Distribution of Results by Composite Performance Index Points] This page of the Report is designed to help schools see how their students are clustered within each performance level and Composite Performance Index (CPI) point interval for each subject area. It can also be used to illustrate how any given school or subgroup with similar aggregate CPI points may have very different percentages of students at each performance level. This results from different distributions of individual scores within CPI point intervals. 1. Performance Levels Arranged by CPI Points Section 1 provides a separate row for each applicable standard and MCAS Alternate Assessment performance level at the grade level of the Report. It also provides a row showing Total CPI Points and one showing Total Students. 2. CPI Points The column in section 2 illustrates how CPI points correspond to performance levels and, for standard tests, to scaled scores (except at grade 3, for which raw scores are reported instead). The number of CPI points shown is awarded for each student whose results fall within one of the performance level categories listed within that CPI point interval. 3. Student Performance by Subject Area Section 3 shows student numbers and percents in each performance level for the school, district, and state. Each subject area’s results are shown in a separate table; tables are presented side by side on the same page. 4. Totals The total number of CPI points awarded, based on the numbers of students in the column above and multiplied by the corresponding CPI point intervals, is shown for the school, district, and state on the top shaded row of section 4. The total number of students counted toward the school, district, or state CPI is shown on the middle shaded row of section 4. The average number of CPI points is equal to the Total CPI Points divided by the Total Students and is shown on the bottom shaded row. C. Subject Area Subscores on Standard Test [Screen shot of Subject Area Subscores on Standard Test] This section of the School Report provides information on student performance in each reporting category for the tested subjects.^3 Each subject area’s results are presented in a separate table. Each reporting category is a strand of the Curriculum Framework for the tested subject area (e.g., Reading and Literature is a content strand of the English Language Arts Framework and an MCAS reporting category). Subject area subscores are given as raw score points and are the only reported MCAS scores based on both common and matrix-sampled items. Multiple forms are created for each MCAS test; unlike common items, matrix-sampled items differ across test forms and are used to report performance on a broader range of specific curriculum goals than is possible using common items only. A school must administer all forms of a test in a grade for accurate computation of subject area subscores. If a school did not administer all forms of a test in a grade, the subject area subscore pages are left blank in the school’s Report for that grade. This situation often arises in schools with small student populations. For each reporting category and item type used within each tested subject area, the subscore pages show the following (in raw score points): • the number of possible points for common items, the number of possible points for matrix items, and the total number of possible points • the average total number of points attained and the percentage of total possible number of points attained at the school, district, and state levels^4 • the average total number of points attained and the percentage of total possible number of points attained for students at the state level in each of the standard MCAS performance levels for the tested grade The following cautions apply only to raw scores, not to performance level results or scaled scores, both of which are statistically equated from year to year. Refer to the 2007 MCAS Technical Report for more information. ^3 Alternate assessment results are not included in subject area subscores. ^4 Because schools may have had different numbers of students taking each test form, average points were determined using a two-step process. First, the average raw score was determined for each unique item. Second, the sum of these average raw scores was calculated. CAUTION: Raw Scores versus Scaled Scores. The numbers for possible subscore points are based on raw score points, not on the scaled scores used elsewhere throughout the Report (except at grade 3, where only raw scores are reported). Since subject area subscores combine results for both common and matrix-sampled items, total points for a subject area may differ on these subscore pages from total raw points reported elsewhere in this Report or in other MCAS reports, which report raw scores for common items only. CAUTION: Longitudinal Comparisons. Comparisons of performance on subject area subscores across years must be made with caution, because the number of items contributing to each subscore is relatively small, and because the difficulty of items may vary somewhat from year to year. The pool of matrix-sampled items varies from year to year as items move from the matrix-sampled sections to the common sections of the tests to replace publicly released common items. D. Performance Level Results [Screen shot of Performance Level Results] Performance level results for the school, district, and state indicate, for each subject area tested, the percent of students performing at each of the four standard MCAS performance levels for the tested grade level.^5 These percentages are based on the performance of all students who took either the standard MCAS tests or the MCAS Alternate Assessment.^6 Performance level results do not include the results of students in the following categories, for whom performance level results were not generated: • Absent (ABS) • Medically Documented Absence (MED) • PRE • First-Year LEP (LEP) • Incomplete (INC) • Transferred (TRN) See pages 24–25 of this Guide for explanations of these six categories. At the school level, the charts include only students who were enrolled on October 1, 2007, unless the school is the only school in its district. ^5 District-level and state-level results include performance level results for students in out-of-district placements. ^6 Alternate assessment performance level results are matched with standard MCAS performance levels in this section. See page 5. E. Three-Year Comparison of School Performance Level Results [Screen shot of Three-Year Comparison of School Performance Level Results] This section of the 2008 School Report shows MCAS results from the spring 2006, 2007, and 2008 administrations. A separate page is generated for each subject area tested. These data, along with results from other past MCAS tests, can be used to make longitudinal comparisons of results from tests that were operational in all three years. Only 2007 and 2008 results are shown on the individual summary report for each of the following high school tests, which were new in 2007: • Biology • Chemistry • Introductory Physics • Technology/Engineering This page is not included for STE in the grade 10 School Report or District Report. For grade 3 tests, no scaled scores are generated. Consequently, the table shown on the right side of the sample on the previous page does not appear in the grade 3 reports. Additionally, since the grade 3 performance level of Above Proficient was used for the first time in 2006, the Proficient performance level data in this section of the grade 3 English Language Arts School Report includes both Proficient and Above Proficient grade 3 student results, to facilitate comparison of data across years. Performance Level Results include students who took either the standard MCAS test or MCAS-Alt. Distribution of Results on Standard Test by Scaled Score Interval includes only students who took the standard MCAS test. F. Performance Level and CPI for Subgroups of Students [Screen shot of Performance Level and Preliminary Composite Performance Index for Subgroups of Students] The information on these pages of the School Report is used for two primary purposes: • to determine how effectively school programs are serving various student populations • to promote fair comparisons among schools and among districts by allowing them to compare the performances of similar student groups A separate page is generated for each subject tested. Each page shows performance level results at school, district, and state levels for students in ten different AYP Subgroups and in seven Other Subgroups, as listed in the graphic on this page. In addition, total number of students is reported, along with average total percent for each performance level, and average number of CPI points awarded, for school, district, and state. Note that, on these pages of the Report, all references to LEP students apply to current LEP students who participated in testing and who generated results. G. Competency Determination Results (Grade 10 Only) [Screen shot of Competency Determination Results] Beginning with the class of 2010, students must reach or exceed the Proficient performance level on the grade 10 English Language Arts and Mathematics tests. They must also reach or exceed the Needs Improvement performance level in one of the four high school Science and Technology/Engineering tests in order to earn a Competency Determination (CD). This section of the grade 10 Report shows the number and percent of class of 2010 students in the school, district, and state who • have attained a CD in all three subjects • have attained a CD in each subject • need to complete an Educational Proficiency Plan (EPP) in English Language Arts and/or Mathematics These CD results are cumulative and include results for students who earned their CD in prior MCAS tests. For complete information on the EPP requirements, including regulations, Commissioner’s updates, frequently asked questions, and resources for planning and implementation see www.doe.mass.edu/hsreform/epp. VII. Understanding the 2008 High School Science and Technology/Engineering Participation and Performance Reports In spring 2008, schools were required to offer a high school MCAS Science and Technology/Engineering (STE) test to students in grades 9 and 10 who were enrolled during the 2007–2008 school year in a course in biology, chemistry, introductory physics, or technology/engineering that was aligned to the Massachusetts Curriculum Frameworks. School, district, and state results for the high school STE tests were reported for the first time this year. There are several unique characteristics of the high school STE tests to consider when interpreting these reports. 1. Participation Requirements Students who were in grade 9 during spring 2008 were not required to take an STE test. They were eligible to participate (and the Department strongly recommended that they participate) if they were enrolled in a corresponding course, but they were allowed to opt out, with the understanding that they would miss an opportunity to meet the high school STE graduation requirement and would be required to take a test in grade 10. Students who were in grade 10 during the spring 2008 MCAS administration were required to take one of the four high school STE tests if they did not particpate in 2007. The Science Participation Summary Reports provide information regarding the grade-level and demographic characteristics of students who participated in each of the four high school STE tests. The participation columns specify, by subgroup, the number and percentage of students who participated in each high school STE test at grades 9 and 10. The percentages are based on March and June 2008 SIMS enrollment data. It is important to keep in mind that, given this information, it is not possible to distinguish the number of grade 9 students who opted out of testing from the number who were taking science courses that did not correspond to one of the four end-of-course STE tests. 2. Variations in Course Offerings and Course-Taking Patterns by School and District Schools and districts often offer more than one science course or sequence of courses (e.g., core, honors, AP). Students in one school who plan, for example, to follow an AP or honors science sequence might be more likely to take biology in grade 9 and chemistry in grade 10. In another school, students following a core sequence might be more likely to take introductory physics in grade 9 and biology in grade 10. For this reason, the MCAS high school biology results for one school or district may represent the performance of grade 9 honors students, while biology results for another school or district may reflect the performance of students who are not following an honors sequence. Because of these variations in course offerings and course-taking patterns, please use caution when comparing district-to-district and school-to-school test results by subject. 3. Comparability The high school Biology, Chemistry, Introductory Physics, and Technology/Engineering tests were developed independently to assess student performance on the learning standards for their respective content areas in the Massachusetts Science and Technology/Engineering Curriculum Framework. While external analyses have established that the four tests have important technical similarities^1 and performance level descriptors for each of the four tests are similar in scope and rigor, the content assessed by each of the four tests is distinct. Therefore comparisons of student performance across these tests should be made cautiously. ^1 The University of Massachusetts, Amherst, study titled “Psychometric Analyses of the 2006 MCAS High School Science Tests” is available at www.mcasservicecenter.com (click “MCAS”). Appendix A: RAW SCORE TO SCALED SCORE CONVERSION TABLES TABLE 1. 2008 MCAS GRADE 4 RAW SCORE TO SCALED SCORE CONVERSIONS Grade 4 English Language Arts Mathematics Raw Score Scaled Score Scaled Score 0 200 200 1 200 200 2 200 202 3 202 202 4 202 204 5 202 204 6 202 204 7 204 206 8 204 206 9 204 208 10 204 208 11 206 210 12 206 210 13 206 212 14 206 212 15 208 214 16 208 214 17 208 216 18 208 216 19 210 216 20 210 216 21 210 218 22 212 218 23 212 218 24 212 220 25 214 220 26 214 220 27 214 222 28 214 224 29 216 224 30 216 226 31 216 228 32 216 230 33 216 230 TABLE 1. 2008 MCAS GRADE 4 RAW SCORE TO SCALED SCORE CONVERSIONS (CONTINUED) Grade 4 English Language Arts Mathematics Raw Score Scaled Score Scaled Score 34 218 232 35 218 234 36 218 234 37 218 236 38 218 238 39 220 240 40 220 242 41 220 244 42 222 248 43 224 250 44 224 254 45 226 258 46 228 260 47 228 262 48 230 264 49 232 268 50 234 270 51 236 274 52 236 280 53 238 280 54 240 280 55 242 56 244 57 246 58 248 59 250 60 252 61 254 62 258 63 260 64 264 65 268 66 272 67 276 68 280 69 280 70 280 71 280 72 280 TABLE 2. 2008 MCAS GRADE 5 RAW SCORE TO SCALED SCORE CONVERSIONS Grade 5 English Language Arts Mathematics Science and Technology/ Raw Score Scaled Score Scaled Score Engineering Scaled Score 0 200 200 200 1 200 200 200 2 202 202 202 3 202 202 202 4 204 204 204 5 204 204 204 6 206 204 204 7 206 206 206 8 206 206 206 9 208 208 206 10 208 210 208 11 210 212 208 12 210 212 210 13 212 212 210 14 212 214 212 15 214 214 212 16 214 216 214 17 216 216 214 18 216 216 216 19 216 218 216 20 218 218 216 21 218 218 218 22 220 218 218 23 220 220 218 24 220 220 220 25 222 220 220 26 224 222 220 27 226 224 222 28 228 226 224 29 228 228 226 30 230 228 228 31 232 230 228 32 234 232 230 33 236 234 232 34 236 236 234 35 238 236 236 36 240 238 236 37 242 240 238 38 244 242 240 39 246 244 242 40 248 246 244 TABLE 2. 2008 MCAS GRADE 5 RAW SCORE TO SCALED SCORE CONVERSIONS (CONTINUED) Grade 5 English Language Arts Mathematics Science and Technology/ Raw Score Scaled Score Scaled Score Engineering Scaled Score 41 250 248 246 42 252 250 248 43 256 252 252 44 258 254 254 45 262 258 256 46 264 260 260 47 268 262 262 48 272 262 264 49 278 264 266 50 280 268 268 51 280 270 272 52 280 274 276 53 280 280 54 280 280 TABLE 3. 2008 MCAS GRADE 6 RAW SCORE TO SCALED SCORE CONVERSIONS Grade 6 English Language Arts Mathematics Raw Score Scaled Score Scaled Score 0 200 200 1 200 200 2 202 202 3 202 202 4 204 202 5 204 204 6 204 204 7 206 204 8 206 206 9 208 208 10 208 210 11 208 210 12 210 212 13 210 212 14 212 214 15 212 214 16 214 214 17 214 214 18 216 216 19 216 216 20 218 216 21 218 216 22 218 218 23 220 218 24 220 218 25 222 218 26 222 218 27 224 220 28 226 220 29 228 220 30 230 222 31 232 224 32 234 226 33 236 228 34 238 230 35 240 230 36 242 232 37 242 234 38 244 236 39 246 238 40 248 240 41 250 242 TABLE 3. 2008 MCAS GRADE 6 RAW SCORE TO SCALED SCORE CONVERSIONS (CONTINUED) Grade 6 English Language Arts Mathematics Raw Score Scaled Score Scaled Score 42 252 244 43 254 246 44 258 248 45 260 250 46 264 252 47 268 254 48 272 258 49 278 260 50 280 262 51 280 264 52 280 268 53 272 54 280 TABLE 4. 2008 MCAS GRADE 7 RAW SCORE TO SCALED SCORE CONVERSIONS Grade 7 English Language Arts Mathematics Raw Score Scaled Score Scaled Score 0 200 200 1 200 200 2 202 202 3 202 202 4 202 202 5 202 204 6 204 204 7 204 206 8 204 206 9 206 208 10 206 210 11 206 210 12 206 212 13 208 212 14 208 214 15 208 214 16 210 214 17 210 216 18 210 216 19 210 216 20 212 216 21 212 218 22 212 218 23 214 218 24 214 218 25 214 218 26 216 220 27 216 220 28 216 220 29 216 222 30 218 224 31 218 226 32 218 228 33 218 230 34 220 232 35 220 232 36 220 234 37 222 236 38 224 238 39 226 240 40 226 242 41 228 244 TABLE 4. 2008 MCAS GRADE 7 RAW SCORE TO SCALED SCORE CONVERSIONS (CONTINUED) Grade 7 English Language Arts Mathematics Raw Score Scaled Score Scaled Score 42 230 246 43 230 248 44 232 250 45 234 252 46 234 254 47 236 256 48 238 258 49 240 260 50 240 262 51 242 264 52 242 268 53 244 272 54 246 280 55 246 56 248 57 248 58 250 59 252 60 254 61 256 62 258 63 260 64 262 65 266 66 268 67 272 68 276 69 280 70 280 71 280 72 280 TABLE 5. 2008 MCAS GRADE 8 RAW SCORE TO SCALED SCORE CONVERSIONS Grade 8 English Language Arts Mathematics Science and Technology/ Raw Score Scaled Score Scaled Score Engineering Scaled Score 0 200 200 200 1 200 200 200 2 202 202 202 3 202 202 202 4 204 204 204 5 204 204 204 6 206 204 204 7 206 206 206 8 208 208 206 9 208 208 208 10 208 210 208 11 210 212 210 12 210 212 212 13 212 214 214 14 212 214 214 15 214 214 214 16 214 216 216 17 216 216 216 18 216 216 216 19 218 216 218 20 218 218 218 21 218 218 218 22 220 218 218 23 220 218 220 24 222 220 220 25 224 220 220 26 226 220 222 27 228 222 224 28 230 224 226 29 232 226 226 30 234 228 228 31 236 230 230 32 238 232 232 33 240 234 232 34 240 236 234 35 242 238 236 36 244 240 238 37 244 240 240 38 246 242 240 39 246 244 242 40 248 246 242 41 250 248 244 42 250 250 246 TABLE 5. 2008 MCAS GRADE 8 RAW SCORE TO SCALED SCORE CONVERSIONS (CONTINUED) Grade 8 English Language Arts Mathematics Science and Technology/ Raw Score Scaled Score Scaled Score Engineering Scaled Score 43 252 250 248 44 254 252 248 45 256 254 250 46 258 258 252 47 262 260 254 48 264 262 256 49 268 262 258 50 272 264 262 51 278 268 266 52 280 270 272 53 276 280 54 280 280 TABLE 6. 2008 MCAS GRADE 10 RAW SCORE TO SCALED SCORE CONVERSIONS Grade 10 English Language Arts Mathematics Raw Score Scaled Score Scaled Score 0 200 200 1 200 200 2 202 202 3 202 202 4 202 204 5 202 204 6 204 206 7 204 206 8 204 212 9 206 214 10 206 216 11 206 216 12 208 216 13 208 218 14 210 218 15 210 218 16 212 218 17 212 218 18 214 220 19 214 220 20 214 220 21 214 220 22 216 222 23 216 224 24 216 226 25 216 228 26 218 230 27 218 232 28 218 234 29 218 236 30 218 236 31 220 238 32 220 240 33 220 242 34 220 242 35 222 244 36 224 246 37 226 248 38 226 248 39 228 250 40 230 252 41 232 254 42 232 256 TABLE 6. 2008 MCAS GRADE 10 RAW SCORE TO SCALED SCORE CONVERSIONS (CONTINUED) Grade 10 English Language Arts Mathematics Raw Score Scaled Score Scaled Score 43 234 256 44 236 258 45 238 260 46 238 260 47 240 262 48 242 262 49 242 262 50 244 264 51 246 264 52 246 264 53 248 266 54 250 266 55 250 268 56 252 270 57 254 270 58 256 274 59 258 278 60 258 280 61 260 62 262 63 264 64 266 65 266 66 268 67 272 68 274 69 278 70 280 71 280 72 280 TABLE 7. 2008 MCAS HIGH SCHOOL RAW SCORE TO SCALED SCORE CONVERSIONS High School Biology Chemistry Introductory Physics Technology/ Raw Score Scaled Score Scaled Score Scaled Score Engineering Scaled Score 0 200 200 200 200 1 200 200 200 202 2 202 202 202 204 3 202 202 202 206 4 204 204 204 208 5 204 204 204 208 6 206 204 206 210 7 206 206 206 210 8 208 206 208 212 9 208 208 208 212 10 210 208 210 212 11 212 212 212 214 12 214 212 214 214 13 216 214 214 214 14 216 214 216 214 15 216 216 216 216 16 218 216 218 216 17 218 216 218 216 18 218 218 218 216 19 218 218 218 216 20 220 218 220 218 21 220 218 220 218 22 222 218 220 218 23 224 220 222 218 24 226 220 224 218 25 226 220 226 220 26 228 222 228 220 27 230 224 230 220 28 232 226 232 220 29 234 228 234 222 30 236 230 236 224 31 236 232 236 226 32 238 236 238 228 33 240 238 240 230 34 242 240 242 232 35 242 240 242 232 36 244 242 244 234 37 244 244 246 236 38 246 246 246 238 39 246 246 248 240 40 248 248 248 242 TABLE 7. 2008 MCAS HIGH SCHOOL RAW SCORE TO SCALED SCORE CONVERSIONS (CONTINUED) High School Biology Chemistry Introductory Physics Technology/ Raw Score Scaled Score Scaled Score Scaled Score Engineering Scaled Score 41 250 250 250 242 42 250 252 252 244 43 252 254 252 244 44 252 254 254 246 45 254 256 256 246 46 256 258 258 248 47 256 260 258 248 48 258 260 260 250 49 260 262 262 252 50 260 262 262 254 51 262 264 264 254 52 262 264 264 256 53 264 266 266 258 54 266 266 266 260 55 266 268 268 262 56 268 270 270 264 57 272 272 272 266 58 274 276 276 270 59 280 280 280 276 60 280 280 280 280 Appendix B: Calculating Threshold Scores (Cut Points) The threshold score is the minimum raw score (number of points) that a student must earn on an MCAS test for her or his performance to be classified at a particular performance level. For example, 18 is the minimum raw score that a student must earn on the 2008 MCAS grade 10 Mathematics test for the student’s performance to be classified at the Needs Improvement level. Because all common MCAS test items are released each year, the test items on the 2008 tests are unique and may vary in difficulty from items on previous tests. To maintain standards and allow comparisons over time, threshold scores representing those standards may change to reflect differences in item difficulty across years. Therefore, MCAS tests are linked across years through a statistical process called equating. Through the equating process, adjustments in the threshold scores for the 2008 MCAS tests maintain the original MCAS standards. THRESHOLD SCORES, MCAS TESTS OF SPRING 2008 Threshold Scores for Performance Levels Grade Subject Area Maximum Advanced Proficient: Needs Score^1 (Above minimum Improvement: Proficient): score needed minimum score minimum score needed needed 3 ELA Reading Comprehension 48 43 36 23 Mathematics 40 36 30 22 4 ELA Composition and Reading 72 63 54 39 Comprehension Mathematics 54 46 39 24 5 ELA Composition and Reading 52 45 36 22 Comprehension Mathematics 54 46 37 23 Science and Technology/Engineering 54 46 38 24 6 ELA Reading Comprehension 52 45 35 23 Mathematics 54 49 40 27 7 ELA Composition and 72 63 49 34 Reading Comprehension Mathematics 54 49 39 26 8 ELA Reading Comprehension 52 47 33 22 Mathematics 54 47 36 24 Science and Technology/Engineering 54 50 37 23 10 ELA Composition and 72 61 47 31 Reading Comprehension Mathematics 60 45 32 18 High Biology 60 49 33 20 School Chemistry 60 47 34 23 Introductory Physics 60 48 33 20 Technology/Engineering 60 54 39 25 ^1 The maximum score is the total number of raw score points that a student could earn in the subject area if he/she answered all of the multiple-choice questions correctly (one point per question) and received the highest possible score for all open-response questions (four points; two points for grade 3 Mathematics), short-answer questions in Mathematics only (one point), and the ELA Composition at grades 4, 7, and 10 (twenty points). Appendix C: MCAS-Related References The resources below are all available on the Department website, at the specific locations listed. MASSACHUSETTS CURRICULUM FRAMEWORKS • English Language Arts Curriculum Framework • Mathematics Curriculum Framework • Science and Technology/Engineering Curriculum Framework • History and Social Science Curriculum Framework Each Framework describes what students should know and be able to do in that subject area at various grade levels. All Frameworks are available at www.doe.mass.edu/frameworks/current.html. OTHER RESOURCES Release of Spring 2008 MCAS Test Items contains all of the common test questions that were used as the basis for student results in the 2008 MCAS reports. Available at www.doe.mass.edu/mcas/testitems.html. Requirements for the Participation of Students with Disabilities in MCAS (Spring 2008 Update) outlines the requirements for participation of students with disabilities and describes allowable test accommodations. Available at www.doe.mass.edu/mcas/participation. Requirements for the Participation of Students with Limited English Proficiency in MCAS, MEPA-R/W, and MELA-O (Spring 2007 Update) outlines the requirements for participation of LEP students. Available at www.doe.mass.edu/mcas/participation. 2007 MCAS Technical Report provides specific technical details about the 2007 MCAS administration and reporting of results. Available at www.mcasservicecenter.com (click “MCAS”). The Massachusetts School and District Accountability System provides an overview of the state’s accountability system. Available at www.doe.mass.edu/sda. Student Records: Questions, Answers, and Guidelines (2002) contains information about the Massachusetts Student Record Regulations. Available at www.doe.mass.edu/mcas/lawsregs.advisories.html. Page 19 of 31