The Massachusetts Board of Elementary and Secondary Education

VOCAL 2018 Summary
September 2018

Background

• The Views of Climate and Learning (VOCAL) survey is designed to provide schools with information on student perceptions of three dimensions of school climate: engagement, safety, and environment. The topics measured within each dimension are summarized below.

  Engagement	Safety	Environment
  The extent students feel the adults/students value diversity, manage dynamics of differences, and avoid stereotypes. The extent students feel there is a social connection and respect between staff/teachers and students, and between students and their peers. The extent students feel engaged intellectually, emotionally, and behaviorally in the classroom, and the extent that students or their parents are engaged in school life.	The extent students feel a bond to the school, and the extent adults/students support the emotional needs of students. The extent that students feel physically safe within the school environment. The extent that students report different types of bullying behaviors occurring in the school and the extent that school/staff/students try to counteract bullying.	The extent that students feel the instructional environment is collaborative, relevant, challenging and supportive of learning. The extent that students have access to systems support that effectively support their social, emotional and mental health well-being. The extent that discipline is fair, applied consistently and evenly, and a shared responsibility.

• The survey is based on the conceptual framework of the U.S. Department of Education's School Climate Surveys.

• The VOCAL survey helps the state to meet requirements included in the Act Relative to Bullying in Schools.

Survey design, responses, scales, and reporting

• Students in grades 5, 8, and 10 were given the option to participate in the survey as part of the state MCAS administration in 2018. The survey was optional for districts, schools, and individual students.

• The Department of Elementary and Secondary Education (DESE) administered 76 items across the three grades with common items across the surveys to allow for later data linking. All items used the same response options: always true, mostly true, mostly untrue, and never true.

• Statewide, 79 percent of eligible students participated in the survey: 84 percent in grade 5, 89 percent in grade 8, and 64 percent in grade 10. Large majorities of schools (94 percent) and districts (97 percent) had at least one student respond. Responding students were demographically similar to non-responding students.

• In addition to reporting individual item responses, DESE developed an overall school climate index score and index scores for the three dimensions of climate (engagement, safety, and environment). These indices were set to a mean of 50 and a standard deviation of 20; differences on the indices of about 3 to 4 points or more represent a meaningful difference in school climate.

• All indices in 2018 are directly comparable to one another and across grade levels. For the first time, districts will also receive a bullying index score that can be compared to other scores. Schools and districts that participated in the 2017 pilot should not compare index scores from 2018 to 2017. The 2018 scores are not anchored on the same scale. The 2018 index scores will form a baseline for future trend data.

• To preserve respondent confidentiality, DESE suppressed data if fewer than 10 students responded, if all students responded identically on an item, or if the reliability of the responses was at least 0.7 on a scale from 0 to 1. After accounting for these criteria, 88 percent of schools and 94 percent of districts received a report.

• Districts and schools will receive reports on the three indices and individual item responses in October 2018 and a second set of reports with subgroup breakouts and a district-level bullying index later in fall 2018.

Findings

• Students generally reported that they experienced positive school climates.

  • In schools with typical school climates, the average student within the school responded "mostly true" to a majority of items and "always true" to a majority of the remaining items

  • Across all three grades, 7 in 10 students responded in the two most positive categories on over 60 percent of the items within each survey.

• Students in grade 5 reported more positive school climates than students in grade 8 or grade 10 across all three dimensions of school climate.

  	Student-level average index score
  	Grade 5	Grade 8	Grade 10
  Overall school climate	58	46	45
  Engagement	56	46	46
  Safety	58	46	45
  Bullying	57	46	45
  Environment	56	46	46

  
  

  • In schools with typical school climates, the average fifth grade student responded "always true" to a 33 percent of the survey's items. In comparison, the average grade 8 and grade 10 student within schools with typical school climates responded "always true" to only 16 percent and 10 percent of the surveys' items, respectively. For example, 22 percent or grade 5 students responded "always true" to the statement, "My teachers use my ideas to help my classmates learn." In comparison, 8 percent and 12 percent of grade 8 and grade 10 students responded similarly to this item.

• Among schools that had sufficient responses to receive reports, the overall school climate index ranged from 27 to 78, showing considerable variation in climate across schools. Substantial variation was also evident among schools for each of the three dimensions of school climate: engagement (scores ranged from 35 to 75), safety (23 to 74), and environment (30 to 73).

• Across the three grades, students in schools with the strongest climates (the top 15 percent of schools) reported many more positive aspects of their school environments when compared to students in the weakest climates (the bottom 15 percent of schools).

  • In grade 5, students in the strongest schools are more open to having inclusive relationships with a diverse range of students and the classroom environment is more collaborative. Teachers have higher expectations for student effort, perseverance, and academics. Students feel safer in the strongest schools with fewer bullying occurrences. They believe their school rules are fair for all students, not just some. Adults are also more active in helping students emotionally and teaching positive behavior.

  • In grade 8, student-on-student relations in the strongest schools are more respectful and open. Teachers have higher expectations for student effort and academics, and use student ideas and interests to plan and guide instruction. Teachers are more readily available when students need help. Students feel safer and can rely more on adults and to a lesser extent students to counteract any bullying. Adults actively promote and model respectful relationships and students can rely to a greater extent on teachers and their peers for emotional and behavioral support. Teachers are more willing to listen to students when they are in trouble.

  • In grade 10, student-on-student relations in the strongest schools are more respectful and open. Teachers have higher expectations for student academics and provide more opportunities for students to challenge themselves. They are more readily available when students need help. Teachers use student ideas, feedback and interests to guide instruction. Students feel safer and can rely more on adults and to a lesser extent students to counteract any bullying. Adults actively promote and model respectful relationships and students can rely to a greater extent on teachers and their peers for emotional and behavioral support. Teachers are more willing to listen to students when they are in trouble.