PACT Logo

Implementation Handbook

Frequently Asked Questions:

1. What is the role of the Embedded Signature Assignments (ESAs) in the PACT TPA?
2. Are there any restrictions on student/intern teaching placements for completing the PACT TPA?
3. What support can be offered to candidates?
4. What are expectations for programs with respect to benchmarking?
5. How is scorer training implemented?
6. What are expectations for implementing scoring?
7. What do programs report to PACT?

1. What is the role of the Embedded Signature Assignments (ESAs) in the PACT TPA?

At the present time, the Teaching Event is the only component of the PACT assessment system that has been approved. Programs with ESAs can use them as additional requirements or as a course assignment. As programs develop ESAs to the point of collecting data, when we have enough scores, a technical group will be convened to review the ESA progress to date and advise PACT on further steps to develop the ESAs to the point that they are a viable component of the PACT assessment system.

2. Are there any restrictions on student/intern teaching placements for completing the PACT TPA?

The candidate must be able to exhibit some degree of discretion to modify the curriculum in the event it is not meeting the students’ needs. Curricula requiring a candidate to ask only scripted questions or read a script are unsuitable placements. If candidates can tailor the script to their students and/or supplement the scripted activities, then they are able to display enough of their own knowledge and skills to be assessed. In the past, very few placements using Open Court and other “scripted” curricula have been so restrictive as to be unsuitable placements.

3. What support can be offered to candidates?

Opportunities to learn the knowledge and skills demonstrated in the Teaching Event should be present in the teaching and learning activities associated with the coursework and field placements. Candidates can be given feedback – by faculty, supervisors, and peers – on drafts of the Teaching Event, following the Guidelines on Assistance to Candidates Completing Teaching Events. Teaching Events that are not complete, i.e., are missing requested information, should not be scored, but returned to candidates to be completed.

4. What are expectations for programs with respect to benchmarking?

Programs are expected to support faculty/supervisor participation in benchmarking and to identify potential benchmarks during the scoring process and to solicit appropriate consent for Teaching Events to be available as benchmarks.

5. How is scorer training implemented?

The training design for which we have demonstrated reliability is a two-day, subject-specific training. Scorers must go through the training to score Teaching Events in the same credential area. They must be trained separately for each credential area scored. Trainings will be developed for “low incidence” subject areas, e.g., Physical Education, Music; in the meantime, scorers in these areas should attend trainings in the area of their choice.
Trainers and scorers should be knowledgeable about the content, content pedagogy, and grade level of the Teaching Events that they score. Programs can either send a representative to be trained as a Lead Trainer, or collaborate with other institutions to collectively acquire a cadre of Lead Trainers. The Lead Trainer oversees the scorers.
At the conclusion of the training, there is a process to check for calibration. Scorers who fail to calibrate must be retrained until they calibrate or else they may not score Teaching Events. In performance assessments, there historically have been a nontrivial percentage of scorers who fail to calibrate initially. In Connecticut, this was 20 % for their licensure assessment.

6. What are expectations for implementing scoring?

Teaching Events that do not meet the passing standard or that are just above the passing standard (i.e., have at least one score of 1) should be double scored. In addition, a random sample of 10 % of the remaining Teaching Events should also be double scored to check reliability.
There should be an appeals process available to students. The appeal is not on the score, but on a failure to follow the procedures for scoring outlined in the PACT Technical Report approved by the California Commission on Teacher Credentialing or subsequent modifications set out in the current Implementation Handbook.
If programs have a writing standard that candidates should meet in any major assignment submitted, the standard can be applied either as a prerequisite to accepting the Teaching Event or alongside the PACT rubrics. PACT will not develop a consortium-wide writing standard, as any writing standard should be applied across major assignments, not just to the Teaching Event.

7. What do programs report to PACT?

PACT collects information from programs to maintain checks of technical quality. In particular, we collect:

  • scores by candidate ID and credential/subject area
  • independent estimates of scorers as to the candidate’s teaching ability (page 2 of the scoring form), and
  • demographic data from candidates to check for fairness.
Programs are given a reporting format (an EXCEL spreadsheet) to provide the first 2 types of information. Candidates complete an online survey to provide the demographic information