Mass.gov
Massachusetts Department of Elementary and Secondary Education
Go to Selected Program Area
Massachusetts State Seal
Students & Families Educators & Administrators Teaching, Learning & Testing Data & Accountability Finance & Funding About the Department Education Board  
>
 
 
>
 
 
 
 
 
 
>
 
 
 
>
 
   

Office of Planning and Research

How do I evaluate and select evidence-based interventions?

The U.S. Department of Education's Non-Regulatory Guidance: Using Evidence to Strengthen Education Investments Download PDF Document provides a useful set of questions districts should consider about research evidence and local capacity when determining which interventions to select. An adapted list of these questions follows.

About evaluating and selecting evidence:

  • What do the majority of studies on this intervention find? Does the intervention have positive and statistically significant effects on important student (or other relevant) outcomes across many studies? Do any studies have null, negative, or not statistically significant findings? If you are conducting your own literature review about a particular intervention, it is essential to pay attention to the complete picture of what the research reveals rather than relying on just a small number of studies.
  • How rigorous are the study designs in terms of the strength of the claim that the intervention caused the outcome obtained, rather than just being correlated with it? Do any meet the criteria for strong or moderate evidence, as defined in ESSA?1 As a point of reference, only randomized control trials and quasi-experimental methods can demonstrate causality.
  • Was the intervention implemented in contexts similar to your own (e.g., similar enrollment size, geography (rural or urban), governance structures, relevant legal or regulatory requirements)?
  • Was the intervention implemented among students similar to those you aim to serve (e.g., English language learners, students more than one year behind in mathematics)?

About local capacity:

  • What resources are required to implement this intervention?
  • Will the potential impact of this intervention justify the costs, or would more cost-effective interventions accomplish the same outcomes?
  • What is the local capacity to implement this intervention? Are funds available? Does staff have the needed skills? Is there buy-in for the intervention?
  • How does this intervention fit into larger strategic goals and other existing efforts?
  • How will this intervention be sustained over time?

ESE will continue to update and expand this page to provide districts and schools with greater access to information about evidence-based practices from here in Massachusetts and nationwide. Please send any suggestions or comments to kwinner@doe.mass.edu.

 

Note:

line



Last Updated: November 8, 2017
E-mail this page| Print View| Print Pdf  
Massachusetts Department of Elementary and Secondary Education Search·Public Records Requests · A-Z Site Index · Policies · Site Info · Contact ESE