Entities Evaluating Teacher Preparation Programs:
Media and Other Independent Organizations
Purposes of TPP Evaluations
read more
❯
Existing TPP Evaluations
read more
❯
Sources of Evidence
read more
❯
How to Develop a TPP Evaluation
read more
❯
About the Report
❯
MEDIA AND OTHER INDEPENDENT ORGANIZATIONS
Media outlets, advocacy groups, and other independent organizations develop rankings and ratings of higher education programs to inform potential students, their parents, and other consumers about the programs’ quality. By rankings, we mean instances whereby organizations collect information about higher education programs, assemble it into an index, and rank the programs in order (first, second, third, and so on). Ratings are similar, in that organizations collect various types of information from higher education programs, but rather than being listed in rank order, the institutions are placed into performance categories based on how well they meet certain criteria.
Evidence
Evidence used by the Media and Other Independent Organizations to assess TPP quality:
Input Measures
- Admission requirements, course offerings, syllabi, textbooks, student teaching observation instruments
Inferences
An evaluation is intended to produce information that can be used to draw reasonable inferences about the quality of programs. By inferences, we mean interpretations or findings based on the above evidence.
- Rating of programs according to performance categories (more nuanced information than pass/fail)
- Extent to which program requirements and course materials align with standards held by the organization
INCENTIVES FOR TPPs
Incentives for TPPs are tangible or intangible reward or sanction tied to the results of an evaluation.
- Prestige associated with being highly rated, which may draw stronger faculty and students, as well as support from state policy makers
Likely Consequences
Each type of TPP evaluation system relies on somewhat different evidence that can be used to draw different inferences. Each system also creates different incentives and consequences for TPPs.
Intended
- Improvement due to increased competition and desire to improve ratings
Unintended
- Overreliance on syllabi may create perverse incentives (e.g., to create impressive syllabi that do not reflect actual instruction)
Mapping the Approach to Purposes
Each type of TPP evaluation system relies on somewhat different evidence that can be used to draw different inferences. Each system also creates different incentives and consequences for TPPs. Thus, instead of asking which TPP evaluation approach is best, the more appropriate and important question is, how well does each approach serve a particular purpose?
Accountability and Monitoring
+ Ratings enable comparisons across participating TPPs on a useful but limited set of attributes
~ Reviewers are often from outside the teacher preparation community
– Methodological constraints in transforming substantive information from comparative ratings into unidimensional rank orderings
Consumer Information
+ Suggests areas of strength and weakness relative to other programs
– Ratings lack credibility, trustworthiness, especially if they are transformed (tacitly or via statistical formulas) into rankings
Program Improvement
+ Suggests areas of strength and weakness relative to other programs
– Ratings lack credibility, trustworthiness, especially if they are transformed (tacitly or via statistical formulas) into rankings