In our effort to support the CAP 2015-16 Pilot, the Massachusetts Department of Elementary and Secondary Education (DESE) has built this document to provide clarifying answers to questions received from our teacher preparation programs and K-12 district colleagues. These clarifications will be included in future editions of the Guidelines for the Candidate Assessment of Performance . DESE will be collecting feedback throughout the CAP Pilot, and will be updating this FAQ periodically on our website. If you have any questions, please email us at EducatorPreparation@mass.gov .
Q: How many observations do supervising practitioners conduct in CAP?
Supervising practitioners are required to conduct a minimum of three observations for candidates. This includes the first announced observation, the first unannounced observation, and the second unannounced observation. Through these observations, CAP emphasizes the importance of the supervising practitioners' perspective and input into decisions about candidate readiness. Both the program supervisor and supervising practitioner may conduct additional observations as needed or desired.
Q: What are the expectations for adhering to the timelines provided in the Guidelines, specifically regarding providing feedback to a candidate within 1-2 days after an observation?
Specific timelines for providing candidates with feedback after an observation are outlined in the Model Observation Protocol, which is an optional component of the Guidelines. Sponsoring Organizations should create a feedback dynamic that best supports candidates in improving their practice and ensuring they are ready day one. It is important to the process that the assessor have sufficient opportunity to analyze evidence after an observation, in order to provide rich, targeted feedback to the candidate.
Q: Do all 6 of the Six Essential Elements need to be rated for both the formative and summative assessments?
Yes. In order to ensure there is sufficient evidence for the summative rating and in order to provide the candidate with an accurate picture of their performance to date, all elements, for each readiness dimension (quality, consistency, and scope), should be rated. Half-way through the practicum, the Program Supervisor and the Supervising Practitioner will have a conversation to calibrate Formative Assessment ratings. In order to do so, they should consider all available evidence collected up to that point, including Observation Forms (Appendix D), student feedback surveys, and other evidence gathered to provide one set of feedback to the candidate. During the Formative Assessment three-way meeting the program supervisor, supervising practitioner, and the candidate meet to review and discuss the ratings, and make adjustments to the candidate's goals and plans as needed, including potentially modifying the goal, increasing support, scheduling more observations, etc.
Q: Do all of the conversations between the PS and SP need to be in person?
No. Although all Three-Way Meetings should be conducted in person, calibration conversations and observation post-conferences, may be conducted over the phone, email, video conference, etc. Sponsoring Organizations should work to develop best practices in support of these conversations and may choose to establish expectations beyond those set forth by ESE.
Q: Will DESE be releasing exemplars for CAP?
Although we do not plan to release exemplars, DESE is working to develop a suite of resources that focus on successful implementation and modeling of effective practice on our website. We have released a CAP workshop on observations and providing meaningful feedback, and will be releasing an additional workshop on using the CAP Rubric. In addition to the resources to support CAP, DESE has also developed a substantial set of materials to support the MA Educator Evaluation Framework including the TEEM video series that focus on high quality implementation of goal-setting, feedback, calibration, and more.
Q: There are several forms embedded within the CAP Guidelines, which ones are required?
ESE has developed an assessment that helps ensure alignment with the MA Educator Evaluation Framework and consistently documents candidates' readiness. Therefore, a certain level of standardization is expected. The following table outlines the various forms and their required use in the process. There are three distinctions among the use of different forms:
It should be noted that for purposes of the CAP Pilot, DESE would value feedback on implementation of the full suite of materials, both required and recommended.
CAP Forms and Resources
Q: What forms should be kept after a candidate completes CAP?
Sponsoring Organizations should keep all required forms in candidate files. Sponsoring Organizations should refer to the Records Retention Advisory for additional information. Additionally, organizations are encouraged to consider the Review Criteria and determine whether documentation associated with CAP may be valuable evidence to demonstrate criteria are being met.
Q: Can documents from the CAP Guidelines be edited?
For all required forms, Sponsoring Organizations may add to documents as needed, but should not remove content from documents. If a form is optional (e.g. Model Observation Protocol), you may revise as you see fit. DESE will continue to collect feedback on the utility and structure of the forms and revise accordingly.
Q: Are observation forms required as is?
Yes. See answer above. It is anticipated that other tools will also be used to actively collect evidence during observations. The forms provided are designed to reflect the synthesis and analysis of the evidence collected relative to the focus elements and a summary of feedback provided to the candidates. DESE has provided examples of tools that could be used in active evidence collection in Appendix D of the Guidelines.
Q: Can the coding be changed on the elements for the various forms (for example, changing 1.A.4 to #1)?
We recommend that sponsoring organizations maintain the coding as currently provided. The reason why we used the coding from the PSTs is to reinforce the Six Essential Elements' connection to the standards and emphasizing the alignment to the Educator Evaluation Framework. By maintaining this nomenclature, DESE hopes to ground candidate experience in the broader PSTs and provide candidates with a clear link to the expectations they will encounter when they enter the workforce.
Q: What are the expectations regarding evidence documentation, especially regarding additional artifacts to support ratings?
In general, assessors will write a summary of any additional artifacts, such as lesson plans and video recordings that are collected to support ratings of essential elements. These summaries should be included in the evidence text box provided on the applicable rubric for the essential element(s) that the artifact has been collected to support. Sponsoring Organizations should support assessors and candidates in using additional artifacts to support rating determinations on the formative and summative assessment. Any candidate artifact that informs assessors' overall ratings should be cited, along with all other pertinent evidence, in the rating rationale section of the CAP rubric. Sponsoring organizations should outline to assessors and candidates their expectations regarding evidence gathering protocol.
Q: Are sponsoring organizations required to send candidate documentation from CAP to DESE licensure department?
No. Sponsoring organizations are requiredto keep records of CAP for each candidate but are not required to send documentation to ESE. Electronic endorsement through the Early ID system serves as evidence that candidates have met all program requirements, including meeting readiness thresholds on CAP. This is because CAP is a requirement for program completion, not for licensure. For a full overview of what is required for record retention, please see the Candidates Record Retention Advisory .
Q: What are the expectations for assessing specialist teacher licenses (Reading Specialists, Academically Advanced, and Speech, Language, and Hearing Disorders) candidates with this new assessment system?
According to regulations (603 CMR 7.07), specialist teacher candidates, which include Reading Specialists, Academically Advanced, and Speech, Language, and Hearing Disorders are not currently required to complete CAP; instead, programs are responsible for designing and implementing their own performance assessment that measures a candidate's ability to demonstrate Subject Matter Knowledge (SMKs) and/or Professional Standards for Teachers (PSTs) as applicable to the license. For example, the Reading Specialist requirements are subject matter requirements and therefore the assessment should measure readiness against those expectations. Many Reading Specialist programs are structured to screen for performance on the PSTs at admission (by requiring an initial license and/or demonstrated proficiency on the MA Educator Evaluation Model Rubrics ). When creating their performance assessment for specialist candidates, Sponsoring Organizations may choose to mirror CAP in structure and protocol, while modifying to assess the specialist candidates on the criteria required of that license (603 CMR 7.07). Candidates must be observed a minimum of three observations throughout the practicum. A modified version of Appendix A: CAP Form and Rubric from the Guidelines to document the practicum observations and meetings has been created for program use, and is available on the CAP website.
Q: Do add license candidates need to participate in the CAP Pilot?
No. At this time, CAP is not required for candidates seeking to add a license in the following areas (these licenses require a field-based experience): early childhood, English language as a second language, or elementary teacher, as teacher of students with moderate disabilities, teacher of students with severe disabilities, teacher of the deaf and hard-of-hearing, or teacher of the visually impaired. If candidates completed a dual program where they are endorsed by the Sponsoring Organization for two licenses, CAP must be completed in both fields. See the Add License vs. Dual Licensure Advisory for more information.
Q: What are the expectations of programs regarding participation in the 2015-16 CAP Pilot?
The 2015-16 CAP Pilot requires all Sponsoring Organizations with teacher preparation programs to pilot the assessment with 10 or more completers. This is a minimum requirement; the threshold of 10 completers may be cumulative throughout the academic year of 2015-16, inclusive of both fall and spring semesters. If you are a Sponsoring Organization with less than 10 completers total, you must complete the pilot with all candidates completing in 2015-2016. The 2015-2016 year was designed as a pilot to allow for adjustments to be made to the assessment and to ensure that Sponsoring Organizations could adequately prepare for the full implementation in 2016-2017. This timeline also aligns with expectations for implementation of the Professional Standards for Teachers . Sponsoring Organizations should consider piloting CAP to the extent that it will be most impactful in preparing for full implementation with all teacher candidates beginning September 1, 2016. You do not need to notify DESE of the candidates participating in the CAP pilot now; participation data will be collected in the spring. For more information, please refer to the memo send on June 30th to Sponsoring Organizations announcing the CAP Pilot.
Q: How do candidates with split dual practicum complete CAP?
For candidates participating in a practicum where hours may be split across placements (e.g., Early Childhood), providers should adjust the CAP process in one of the following ways described below. DESE recognizes the logistical challenges associated with implementing the assessment in these situations and so have outlined two options for implementation. Sponsoring Organizations should set policies within their programs relative to these two options that ensure a high-quality, consistent assessment of candidate readiness. The two options are:
Q: What incentives can Sponsoring Organizations offer to supervising practitioners?
Supervising practitioners may earn up to 15 Professional Development Points (PDPs) annually for supervising a teacher candidate during his/her practicum. Sponsoring organizations may offer additional incentives to supervising practitioners for participation such as credits toward a course at your Organization. CAP, relative to the PPA, does require supervising practitioners to take a more active role in the evaluation of candidates placed in their classrooms. Because of the alignment of CAP to the Educator Evaluation Framework, serving as a supervising practitioner can be mutually beneficial for both the candidate and teacher. Not only are supervising practitioners uniquely positioned to serve in their role given the familiarity with their own evaluation system, but they also stand to benefit from being on the assessor-side of the evaluation process and opportunities to strengthen and demonstrate their own professional practice. DESE recommends that Sponsoring Organizations leverage the increased engagement of supervising practitioners as its own natural incentive. The majority of supervising practitioners indicate that they serve in the role because of a desire to support the next generation of teachers, CAP in fact gives them a greater, more concrete voice in that endeavor.
Q: Do SO's need to report to the DESE how they are participating in the pilot?
No. Sponsoring organizations will not need to report to DESE how they will be piloting CAP; however, we will be collecting formative and summative assessment ratings of teacher candidate participating in the pilot from each Sponsoring Organization. DESE has created an Excel spreadsheet for Sponsoring Organizations to complete by the end of the pilot year (May 30, 2016) and was released in November 2015. This spreadsheet collects basic information of involvement, as well as formative and summative ratings of each candidate who participated. If Sponsoring Organizations are participating in the pilot of the online platform, you can use that as the mechanism for collection and reporting the formative and summative assessment rating data to ESE.
Q: What are the expectations for administering the model DESE Student Feedback Surveys?
As stated in the Guidelines, student feedback must be collected using the model DESE Student Feedback Surveys, because they align to the Standards for Effective Teaching Practice. Although we strongly encourage consistency during the CAP Pilot, we understand that not all districts use these surveys. In these instances, we recommend use of the short form surveys and we will also be releasing micro forms of the surveys in 2016 that can be used to collect student feedback.
Q: Does DESE have information regarding the reliability and validity of the DESE Model Feedback Surveys?
Yes. The DESE Model Feedback Survey items were piloted by Massachusetts students and administrators—over 10,000 Massachusetts students and 1,500 Massachusetts educators piloted survey items in 2013-14—and were evaluated across four aspects of test validity: content, substantive, structural, and generalization validity. To see results from the student survey validation work from the pilot, download the full technical report. A brief write-up of the pilot and instrument development process Page 2.
Q: How do teacher candidates collect feedback from students with special needs?
From the Student Survey for Classroom Teachers DESE Model Survey Administration Protocol :
Collecting feedback from students with special needs is a valuable part of the evaluation process. Districts should make every effort to include all students, or a representative sample of all students, in their feedback collection. When students with disabilities participate in the Model Surveys, their accommodations must be consistent with IEPs and 504 Plans. Since the purpose of the Model Survey is to solicit students' opinions, accommodations can be more intensive than on instruments designed to assess student knowledge and skills.
This may mean very intensive adaptations of the survey content, depending on student need.
In addition, the Educator Effectiveness Guidebook for Inclusive Practice, includes adapted items for the Massachusetts Model Student Feedback Survey.
Q: How is DESE collecting feedback from Sponsoring Organizations as they participate in the CAP pilot?
ESE will be sending surveys to Sponsoring Organizations in December and April to collect feedback from preparation programs, program supervisors, supervising practitioners, and candidates. Please keep track of your feedback throughout your pilot experience, so that we can collect your comments and suggestions for improvement when feedback surveys are released. In addition DESE will be conducting focus groups following the fall and spring pilot to collection additional feedback.
Q: What is the expectation around the forthcoming online platform?
ESE is developing an online platform that will allow assessors to communicate online, upload their evaluations and feedback in real time, and provide their formative and summative ratings electronically, and to easily submit individual candidate data to DESE annually. The platform will be piloted in the spring of 2016 with approximately 700 candidates. The online platform user guide as well as more information on the pilot will be available on our website in December.
Q: What are the expectations of Sponsoring Organizations regarding training and support in the CAP pilot?
It is the responsibility of the Sponsoring Organization to communicate involvement with CAP, as well as provide applicable training and support surrounding CAP. One example of an expectation in our program review criteria includes: Supervising Practitioners and Program Supervisors receive training, support and development from the SO that impacts candidate effectiveness. DESE has created resources to support Sponsoring Organizations in the training of their program supervisors and supervising practitioners participating in CAP.
Last Updated: December 10, 2015
Massachusetts Department of Elementary and Secondary Education 135 Santilli Highway, Everett, MA 02149
Voice: (781) 338-3000 TTY: (800) 439-2370
Disclaimer: A reference in this website to any specific commercial products, processes, or services, or the use of any trade, firm, or corporation name is for the information and convenience of the public and does not constitute endorsement or recommendation by the Massachusetts Department of Elementary and Secondary Education.