Collecting, Interpreting & Utilizing DIBELS Data John Hintze, Ph.D. William J. Matthews, Ph.D. University of Massachusetts School Psychology Program hintze@educ.umass.edu shamrock@educ.umass.edu 413-545-1192 Massachusetts Reading First June 9 & 10, 2004 Christopher Parker, Ph.D. Judith Loughlin, Ph.D. IDEAL Consulting Services, Inc. (508) 636-6615 (866) 254-6136 cparker@idealconsultingservices.com jeloughl@educ.umass.ed www.idealconsultingservices.com Schools as “Host Environments” Each school in which you collect DIBELS data is a unique setting •Varying conditions •Unique set of resources Data collection will need to be modified to best utilize the resources at each particular school Who Can Assist in Data Collection? Most people, after training in administration and scoring of DIBELS, can successfully collect the data. •Use the “Assessment Integrity Checklists” included within the Appendix to determine readiness Data collectors can be…teachers; principals; speech, Title I and SPED personnel; paraprofessionals; school psychologists; PE, art and music teachers; practicum students; parents and volunteers. How can the data be collected? Four possible approaches: •In-class approach •One Day SchoolwideApproach •Multiple Day SchoolwideApproach •Within-Grade Approach •Other: Variation or some combination of the above Selecting an approach… How many students need to be assessed? What types oftime constraintsare you dealing with? •See the “Data Collection Conversion Table” included within the Appendix How many people have been trained and are available to participate? Do you have enough materials (e.g., stopwatches, clipboards, examiner booklets, etc.)? What types of space is available? When do you collect the data? Typically…Fall, Winter and Spring When to schedule the last benchmarking… •At the end of the year? •Earlier, so that the data can be used for instructional decision making? How to Make Sure the BenchmarkingGoes Smoothly: Advanced Planning! See the “Data Collection Checklist” included within the Appendix In addition… •Secure advanced confirmation from the building principal about the approach, testing location, personnel involved and timelines •Confer with teachers about scheduling •Have materials purchased, organized, color-coded, labeled, etc. Materials Booklets (labeled with student information and boxed by classroom) Stimulus materials Class rosters Clipboards Pencils RELIABLE Stopwatches Other Considerations Have a means to ensure correct scoring…that is, make sure the data are being collected reliably!!! Have a plan for following up on… •absentees •students who performed differently from what was expected Have a plan for entering the data and reporting the results Now for the real fun! Once your data are gathered, scored, checked, loaded and printed out in useful form, you are ready to… USE THE DATA TO INFORM INSTRUCTION! “And that’s what it’s all about!” After Benchmarking: Collaborative Grade Level Teams •Look at Datato get a feel for the classroom and individuals Determine the degree to which the data reflects teacher perception Clarify any anomalies •Use the data to create differentiated instructional groupings •Use the data to Assessthe specific skills deficitsof individual students Educational Assessment •Educational Assessment should be Model Driven Why??? Will create a framework for identifyingand improvingstudents’ skill deficits that is… Timely Efficient Effective How??? Problem Solving Approach Steps within the Problem Solving Approach •Identify the Problem Peer-Referenced Assessment Survey Level Assessment •Develop Meaningful Goals •Design Achievement Support Plans •Regularly Monitor Student Progress How is a Problem determined? •A problemis defined as the difference between what is expectedand what occurs. •What is occurring? Examine the student’s performance level via benchmark testing. •What shapes our expectations? Established Benchmark Levels Established Competency Standards Norms Classroom Building District National Shaping Expectations… Interpreting DIBELS Benchmark Levels •Risk Status The following criteria outline the student’s risk status: Low Risk Some Risk At Risk Considerations when using benchmarks •Benchmark standards help delineate the fluency levels that students need to achieve to establish the odds in their favor for achieving subsequent early literacy and/or reading goals. For instance, the odds for achieving subsequent early literacy and/or reading goals for students listed within the low riskcategory are approximately 90%. In contrast, the odds for achieving ensuing early literacy and/or reading goals for students listed within the at riskcategory are approximately 10%. Additional Considerations •Further evidence for trying to help students achieve benchmark standards: Spring-Kind. Spring-Grade 1 At Risk for ISF, LNF & PNF 19 Low Risk for ORF (>40 WRC) Low Risk for ISF, LNF & PNF 92 Low Risk for ORF (>40 WRC) Spring-Grade 1 Spring-Grade 2 At Risk for NWF & ORF 3 Low Risk for ORF (>90 WRC) Low Risk for NWF & ORF 92 Low Risk for ORF (>40 WRC) Shaping Expectations… Interpreting Reading Competency Standards •National Reading Competency Standards (Grades 1-2) The following criteria determine the student’s performance when compared to national reading competency standards: Fluent: greater than or equal to 80 words read correctly Marginally Fluent: between 61 and 79 words read correctly Instructional: between 40 and 60 words read correctly Emerging: between 10 and 39 words read correctly Non-Reader: less than or equal to 9 words read correctly •National Reading Competency Standards (Grades 3-6) The following criteria determine the student’s performance when compared to national reading competency standards: Fluent: greater than or equal to 140 words read correctly Marginally Fluent: between 101 and 139 words read correctly Instructional: between 70 and 100 words read correctly Deficit: between 31 and 69 words read correctly Emerging/Non-Reader: less than or equal to 30 words read correctly Shaping Expectations… Interpreting Reading Competency Standards Example No. 1: Kindergarten Early Literacy Scores [Winter Results] Low Risk = Core Some Risk = Supplemental At Risk = Intensive Example No. 1: Examining the DIBELS Data 1.Are there any students who appear to be in a good position to achieve future benchmarks and overall reading success? What makes you think so? 2.Which students should be targeted FIRST for intensive intervention? Why? 3.Overall, in which skill area were the students most successful? 4.Overall, in which skill area were the students least successful? 5.Take a close look at Flora…what does the data tell us about her early literacy profile? 6.How would you arrange the students into instructional groups based on the data set? Example No. 2: First Grade Early Literacy/Reading Scores [Winter Results] Low Risk = Core Some Risk = Supplemental At Risk = Intensive Example No. 2: Examining the DIBELS Data 1.Are there any students who appear to be in a good position to achieve future benchmarks and overall reading success? What makes you think so? 2.Which students should be targeted FIRST for intensive intervention? Why? 3.Take a close look at Tony…what does the data tell us about his early literacy/reading skills? 4.Take a close look at Meredith…what does the data tell us about her early literacy/reading skills? 5.How would you arrange the students into instructional groups based on the data set? Example No. 3: Third Grade Reading Scores [Winter Results] Some Risk = Supplemental At Risk = Intensive Example No. 3: Examining the DIBELS Data 1.How many students are at riskof demonstrating significant reading difficulties? 2.What criteria would you use to select the students who are going to receive intensive intervention(s)? Is it difficult to do so because so many students are at risk?Does it makes sense to consider other standards of success? 3.Would you want to collect additional data related to the at-risk students’ skills? If so, what kinds of information would be most helpful to you when considering intervention efforts? Example No. 3: Third Grade Reading Scores [Winter Results] Example No. 3: Third Grade Early Literacy/Reading Scores Low Risk / Established = Core Some Risk / Emerging = Supplemental At Risk / Deficit = Intensive Use the Data to Make Educational Decisions for Individual Students •Is there a need to move through the remaining steps of the Problem Solving Approach? Is the target student’s performance significantly discrepantfrom established standards of success? If so, further investigate? How??? Collect Data across Multiple Days Conduct a Survey Level Assessment Collect Data across Multiple Days Summary of Problem Identification Student’s Name: Samuel Rogers Grade: 2 Date: September 9 *Risk status determined from Fall Benchmark Standards for second-grade students. Reading Probes Day 1 WRC/E Day 2 WRC/E Day 3 WRC/E •Passage 1 8 12 11 •Passage 2 7 9 8 •Passage 3 7 11 9 Daily Median 7 11 9 Academic Area Level Day 1 Day 2 Day 3 Median Benchmark Status Does a Significant Discrepancy Exists? Reading 2 7 11 9 9 At Risk* Yes Example 1: Chaz-Kindergarten Dates Collected: January 9-11 Academic Area Curric. Level Day 1 Day 2 Day 3 Median Benchmark Status Does a Significant discrepancy Exist? Phoneme Segmentation Fluency HM N/A 5 9 6 Example 1: Chaz-Kindergarten Dates Collected: January 9-11 *Risk status determined from Winter Benchmark Standards for kindergarten students. Academic Area Curric. Level Day 1 Day 2 Day 3 Median Benchmark Status Does a Significant discrepancy Exist? Phoneme Segmentation Fluency HM N/A 5 9 6 6 At Risk* Yes Example 2: Lauren-First Grade Dates Collected: January 16, 19 & 20 Example 2: Lauren-First Grade Dates Collected: January 16, 19 & 20 *Risk status determined from Winter Benchmark Standards for first-grade students. Academic Area Curric. Level Day 1 Day 2 Day 3 Median Benchmark Status Does a Significant Discrepancy Exist? Nonsense Word Fluency Open Court N/A 13 16 13 13 At Risk* Yes Example 3: Chloe-Second Grade Dates Collected: January 9-11 Academic Area Curric. Level Day 1 Day 2 Day 3 Median Benchmark Status Does a Significant discrepancy Exist? Oral Reading Fluency Harcourt 2 55 49 53 Example 3: Chloe-Second Grade Dates Collected: January 9-11 *Risk status determined from Winter Benchmark Standards for second-grade students. Academic Area Curric. Level Day 1 Day 2 Day 3 Median Benchmark Status Does a Significant discrepancy Exist? Oral Reading Fluency Harcourt 2 55 49 53 53 Some Risk* No Example 4: Larry-Third Grade Dates Collected: January 16, 19 & 20 *Risk status determined from Winter Benchmark Standards for third-grade students. Academic Area Curric. Level Day 1 Day 2 Day 3 Median Benchmark Status Does a Significant Discrepancy Exist? Oral Reading Fluency Open Court 3 99 99 92 99 Low Risk* No What is a Survey-Level Assessment (SLA)? •SLA is the interpretation of “a general sample of the student’s behavior on a given task” (Howell & Kaplan, 1980, p. 17). •SLA with the use of DIBELS & CBM involves assessing students in successive levelsof the general education curricula for the purpose of making meaningful educational decisions. Steps for Conducting a Survey-Level Assessment 1.When conducting an SLA in reading, select a minimum of three CBM/ORF reading probesfrom current and lower grade levels (e.g., Standard Reading Assessment Passagesfrom Edformation). 2.When conducting an SLA in any of the early literacyareas, selectone DIBELS probefrom the pertinent skill domain. 3.When conducting an SLA, administer and score the probes in progressively lower levels until successis reached. Administration of the SLA is discontinued when an appropriate instructional levelhas been reached. 4.Record data in a SLA table and summarize on a SLA graph (optional). Defining Success Successcan be based on either: a)Instructional Placement Level (Reading only) b)Normative Level Instructional Placement Levels •Utilizing passages from Primer through 2nd grade reading materials, a student would be expected to read a minimum of 40-60 words correctly with 4 or fewer errors. •Utilizing passages from 3rd through 6th grade reading materials, a student would be expected to read a minimum of 70-100 words correctly per minute with 6 or fewer errors. Recording and Summarizing the Data Record the student’s scores and medians in a table. SLA data also can be displayed visually by plotting the median scores on a instructional placement level graph. Survey-Level Assessment: Sample using Instructional Placement Levels Text Box: Student: Kate Grade: 2 Academic Area: Reading Dates Collected: Jan. 12-15 Results of SLA Plotting Kate’s Scores on a Instructional Placement Level Graph Using Instructional Placement Levels Example: Hector-Second Grade Student: Hector Grade: 2 Academic Area: Reading Dates Collected: Feb. 9-11 Results of SLA Steps for Writing Meaningful Educational Goals 1.Collect current performance data. •Utilize data collected during Problem Identification & Survey Level Assessment 2.Specify measurement conditions. •Timelines •Measurement materials 3.Specify criterion for success. •Goals and objectives Determining Criterion for Success Using Established Benchmark Standards •Again, refer to the Established Benchmark Standards published by the University of Oregon and use professional judgment to determine a criterion for success. Example: Tammy, a kindergarten student, names 12 letters correctly in one minute during the Winter benchmark testing period. Her score would place her at riskof failing to meet appropriate early literacy and reading trajectories in the future. Kindergarten students who demonstrate appropriate early literacy and reading growth typically name at least 27 letters correct in one minute during the Winter benchmark testing period. Thus, it makes sense to set a short term goal of 27 letters named correctly in one minute for Tammy by the end of the Winter testing period (10-12 weeks). Writing Goals for Tammy Using Normative Growth Rates •An alternative to established benchmark standards is to use the early literacy/reading Normative Growth Rates of elementary-aged students reported in published research studies and technical papers. In these studies, Fall, Winter and Spring benchmark testing results from K-6 students who received standard early literacy and/or reading instruction (i.e., highly specialized curricula or teaching practices were not implemented on a widespread basis) were identified and the rates of progress were calculated. •Example: James, a second-grade student, reads 48 words correctly in one minute during the Winter benchmark testing period. His score would place him at riskof failing to meet appropriate reading trajectories in the future. Research suggests that it is realistic to expect second-grade students to grow 1.5 words read correctly per week. Using this information in tandem with the formula below allows one to determine James’ criterion for success. Criterion for Success = Baseline Score + (Grade Growth Rate x Number of Weeks) James’ Criterion for Success = 48 + (1.5 x 10) = 63 words read correctly Writing Goals for James Expected Rates of Growth: WRC per Week Fuchs, Fuchs, Hamlett, Walz, & Germann(1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22(1), 27-48. Expected Weekly Growth Rates for Critical Early Literacy Skills From C. Parker (2003). [Utilizing general outcome measures to examine the weekly growth rates of kindergartners’and first graders’early literacy skills]. Unpublished raw data. Gd. Subtest Realistic K Letter Naming Fluency 0.93 K Initial Sound Fluency 0.49 K Phoneme Segmentation Fluency 0.92 1 Phoneme Segmentation Fluency 0.64 1 Nonsense Word Fluency 0.87 Example 1: Hanna-Kindergarten Student:Hanna Grade:K Academic Area:Alphabetic Understanding Dates Collected:February 3, 4 & 6 Using Established Benchmark Standards and Normative Growth Rates, determine a long-term goalfor Hanna. In _____ weeks, when given a randomly selected probe from the Letter Naming Fluency subtest, Hanna will name _____ letters correctly per minute. *Benchmark status determined from Winter Benchmark Standards. Example 2: Josh-First Grade Student:Josh Grade:1 Academic Area:Alphabetic Principle Dates Collected:January 12, 16 & 19 Using Established Benchmark Standards and Normative Growth Rates, determine a long-term goalfor Josh. In _____ weeks, when given a randomly selected probe from the Nonsense Word Fluency subtest, Josh will name _____ letter/sound correspondences correctly per minute. *Benchmark status determined from Winter Benchmark Standards. Using Established Benchmark Standards and Normative Growth Rates, determine a long-term goalfor Jared. In _____ weeks, when given a randomly selected passage from Level _____ of the Standard Progress Monitor Passages, Jared will read _____ words correctly per minute. Intervention Planning: Constantly Focus on the Five Big Ideas Phonemic Awareness Oral segmenting and blending Alphabetic Principle Letter-sound matching Segmenting and blending with print Fluency with connected text Vocabulary Development Comprehension Intervention Planning: The Vehicle for Improving Instructional Outcomes Optimal Use of the Core Curriculum Appropriate use of whole group instruction Small group differentiated instruction Use of Supplemental Intervention Programs Use of Intensive Intervention Programs Generic Intervention Planning - Intensify Optimal Use of Core Curriculum by Classroom Teacher Teacher conducts whole group instruction daily Unit kick offs/Teacher read-alouds Vocabulary and comprehension Teacher conducts differentiated small group instruction with each group daily Supplemental/strategic intervention groups are cond Generic Approach to Optimizing Learning: Focus on Instruction Time on task (Academic Engaged Time) Opportunities to respond Make instruction maximally interactive Quick corrective feedback Materials targeted to the student’s skill needs and instructional level Teach intentionally Teach explicitly Teach specifically - target gaps Teach intensively - frequency & duration Teach relentlessly Generic Approach to Optimizing Learning: Focus on Instruction Strategies for Progress Monitoring •How Frequently is Progress Monitored? How frequentlystudents are tested should be related to the severityof the problem. A balance must be struck between what is ideal and what is feasible. The less frequently one assesses, the “higher quality”the information should be. In closing… Text Box: The greatest danger for most of us is not that our aim is too high and we miss it, but that it is too low and we reach it. Resources •DIBELS http://dibels.uoregon.edu/ •Edformation/AIMSweb http://www.edformation.com/ •National Center on Student Progress Monitoring http://www.studentprogress.org/ •Institute for the Development of Educational Achievement http://idea.uoregon.edu/ •Intervention Central http://www.interventioncentral.org/ ©IDEAL Consulting Services, Inc. 2003-2004