Synthesizing Psychometrics and Causal Inference: Applications of Latent Variable Models to Treatment Heterogeneity, Psychological Networks, and Learning Transfer

Joshua Gilbert

Evaluating the effectiveness of educational interventions presents two distinct challenges: measurement of outcome variables through psychometrics and estimation of program impact through causal inference techniques. Education research has been unnecessarily constrained by the separation of these two domains, and their synthesis is a promising area of methodological research that can inform ongoing substantive debates in the field. I contribute to this synthesis through three dissertation studies. In Study 1, I demonstrate how the common approach to estimating treatment heterogeneity on test score outcomes using interaction effects is susceptible to bias if treatment effects are correlated with item easiness. I show how analysis of item-level treatment effects can eliminate the bias. In Study 2, I explore network psychometrics, in which psychological traits are considered complex systems rather than unidimensional continua. While common in other fields, network models are rare in education research, in part due to computational constraints. I demonstrate how to leverage item response theory modeling approaches to make inferences about causal effects on network structures when direct estimation is not possible, and apply the proposed approach to randomized controlled trials in education. In Study 3, I extend an analysis of a content literacy intervention with two years of new data to examine the persistence of effects over time and potential mechanisms of interdisciplinary learning transfer from language to math using latent mediation analysis. I conclude by summarizing the implications of these studies for education research.

About Joshua Gilbert

Joshua Gilbert is a PhD candidate in Education Policy and Program Evaluation at the Harvard Graduate School of Education, where he works with James Kim and Luke Miratrix. His research interests include the intersection of causal inference and psychometric methods. His work has been published in journals such as Developmental Psychology, Journal of Educational Psychology, Journal of Educational and Behavioral Statistics, Behavior Research Methods, Journal of Research on Educational Effectiveness, Applied Measurement in Education, Psychological Methods, and others.