Why is the Department of Elementary and Secondary Education (ESE) including a questionnaire at the end of grades 5, 8, and 10 MCAS tests?
ESE has included a student questionnaire as part of the MCAS test administration in grades 8 and 10 for the last 15 years. The content of these questionnaires has varied over the years and has included topics such as students' plans after high school, their experiences using technology in the classroom and at home, and their perceptions about their core academic classes.
Why is ESE using the MCAS questionnaire to pilot school climate questions?
One of ESE's strategic priorities is to support students' social-emotional learning, health, and safety, but the agency currently collects little systematic statewide data to assess progress on these issues. A wide range of stakeholders, including educators, parents, advocates for children, and other groups, have asked us to gather more data in this area. We are piloting questions related to school climate to begin developing such measures and to begin determining how we can appropriately use them.
Who will participate in the MCAS questionnaires?
MCAS student questionnaires will be administered to students in grades 5, 8, and 10.
When and how will the MCAS questionnaires be administered?
Students in grades 5 and 8 will complete their questionnaires after completing their MCAS Science and Technology/Engineering tests in April/May. Grade 10 students will complete their questionnaires after taking their MCAS Mathematics test in mid-May. Students will respond to the survey questions and mark their answers in their answer booklets. Questionnaire answers do not affect students' MCAS scores.
What will be measured in the school climate portion of the questionnaire?
Students will respond to questions related to three sub-dimensions and nine indicators of school climate. The sub-dimensions and indicators are:
- engagement (relationships with teachers/staff and with other students, participation in the school community, and cultural and linguistic competence);
- safety (physical safety, emotional safety, and bullying); and
- environment (instruction, mental health, and discipline).
Where did the school climate questions come from?
Questions were drawn from several publicly available resources. Three major sources were the ED School Climate Surveys developed by the U.S. Department of Education, the Program for International Student Assessment (PISA) student survey, and ESE's model student feedback surveys.
What other topics will be covered on the questionnaires?
The grade 8 and 10 questionnaires will include questions about experiences using a computer for school work. Grade 8 will also include questions on time spent on homework, while grade 10 will include questions on plans after high school and on instructional practices in their mathematics classes. The grade 5 questionnaire will only include school climate questions.
How is the questionnaire designed?
The grade 5 questionnaire includes 24 questions; grade 8, 37 questions; and grade 10, 38 questions. Beyond the topics listed above in question 6, students will respond to a set of common school climate questions that appear on all three grade-level questionnaires and to a set of questions that is unique to each grade. Across the three questionnaires, ESE will try out 71 school climate questions in total.
Who participated in the development of the school climate questions?
Over 150 students from the state Student Advisory Council and ESE's five regional student advisory councils reviewed and revised the questions for the surveys. We pilot-tested questions with diverse groups of grade 5 students to ensure the questions were at an appropriate reading level and were understandable. Members of Massachusetts Advocates for Children and the Safe and Supportive Schools Commission also provided feedback on drafts.
May schools and districts choose to not participate in the questionnaires?
Schools and districts are encouraged to participate in the questionnaires to support good data being available statewide as well as for districts. After reviewing the questions, a district or school may choose not to administer the questionnaires, based on their local policies for surveys and research.
May students choose to not participate in the questionnaire?
Student participation is voluntary. ESE recommends that schools and districts follow their usual policy for administering surveys to their students.
How long will the questionnaire take?
Based on pre-testing with students, it should take a student 10 to 15 minutes to complete a survey.
How will ESE use the data?
ESE will conduct a rigorous psychometric analysis and validity study for each questionnaire. This will determine which school climate questions function well and whether they are measuring what we had intended them to measure. ESE will also conduct research to assess if students' perceptions of their school climate are related to their achievement, attendance, and other outcomes.
Will schools and districts receive a report of their results?
Yes. Schools and districts will receive an aggregated summary report of their students' responses to each item, along with comparisons to statewide results. ESE intends for schools to use the data to better understand how students perceive their learning environment.
Are students' individual identities protected?
Yes. Schools and districts will only receive student data in aggregate. Results for questions with fewer than six students responding will not be reported to schools and districts.
Will the questionnaire results for schools and districts be posted on ESE's webpage?
No. Schools and districts will only receive their results through their dropboxes. A technical report and a descriptive report of the state's overall results will be posted on the Office of Planning and Research's webpage.
Will the data provided to schools and districts be used for accountability purposes this year?
No. We will not make any decisions about whether to include these data in our school and district accountability system until we see how the questionnaire performs and whether it generates valid, reliable results over time.