Methodology for Studying Treatment Effect Heterogeneity in Education

Luke Miratrix

An understanding of how much variation treatment variation there is in an evaluation, and what predicts it, is integral to policymakers as they attempt to use findings from randomized trials to form and expand policy. Frustratingly, while so seemingly important, treatment effect heterogeneity is difficult to detect, difficult to predict, and difficult to model. This project seeks to use a blend modern causal inference, machine learning methods, and classic statistical tools from survey sampling to extend existing approaches for treatment effect variation into education contexts. The goal is to generate three methodological tools to best answer several specific policy-relevant questions that pertain to treatment variation in three distinct education contexts (an early childhood center intervention, a middle-school teacher professional development course, and a large district-run high school lottery). These tools are (1) an improved method for detecting the presence of treatment variation by extending current methods to incorporate covariates in order to increase power, (2) a method for using machine learning to predict the units most susceptible to treatment by identifying the likely axes of maximal variation in treatment, and (3) a method for constructing models for treatment variation in order to separate contextual variation as compared to program variation in multi-site trials. This work will provide concrete, accessible methods for taking full advantage of covariates to increase power while, as much as possible, maintaining low model dependence in order to preserve the validity of the original randomized experiment.

About Luke Miratrix

Luke Miratrix is an assistant professor in the Harvard School of Education and affiliate faculty in the Harvard Department of Statistics. His primary research focus is on causal inference methods. In particular, much of his work is on developing methodology to assess and characterize treatment effect heterogeneity in randomized clinical trials and observational studies, and on characterizing variation in treatment impact on post-treatment or latent subgroups such as from non-compliance. He also works on these concerns in the statistical evaluations of cluster-randomized and multi-site trials. Other research interests include data mining using high-dimensional and sparse (regularized) methods, with a focus on text summarization and causal inference with text in contexts such as newspaper corpora, legal decisions, and databases of free-text reports. In his work, his main concerns are usually related to the applicability, interpretability, and legitimacy data-driven arguments, which generally leads him to examine the performance and usability of simple, minimal-assumption methods. Luke Miratrix received his Doctorate in Statistics from University of California, Berkeley in Spring, 2012 after switching to that field in 2009 from SESAME, a doctorate program in Mathematics and Science education also at Berkeley. He also has a MS in Computer Science from M.I.T., a BS in Computer Science from the California Institute of Technology, and a BA in Mathematics from Reed College. Between graduate careers, he was a high school teacher and tutor for 7 years