Beyond the Final Score: Leveraging Process Data for Time-Varying Separable Effects and Accurate Ability Estimation in Education

Youmi Suk

The widespread transition from paper-based to digital assessments in education has generated vast amounts of log data, known as response process data. These data capture real-time information about students' interactions with digital devices, including their evolving performance over time. However, most prior work has focused on static final scores, overlooking the time-varying nature of students' test outcomes. To address this gap, this project proposes two studies that incorporate process data from digital assessments. The first study will develop a framework and methods for evaluating the time-varying separable effects of an educational intervention at multiple time points during a testing period. As an alternative to traditional causal mediation analysis, separable effects analysis investigates the unique effects of individual components (e.g., extra time, a separate session) that make up a broader intervention (e.g., extended time accommodations). The second study will develop a psychometric method that integrates functional outcomes (e.g., evolving test scores) into ability estimation to improve accuracy. Both studies are motivated by data from the National Assessment of Educational Progress (NAEP) and will be demonstrated using process data from the 2017 NAEP digital mathematics assessment. Overall, this project expands the utility of process data in education by leveraging time-varying and functional outcomes and developing new tools for both program evaluation and accurate assessment. With these proposed approaches, I hope to promote more holistic evaluations of educational interventions on student outcomes and more reliable assessments of students' abilities in digital testing environments.

About Youmi Suk

Youmi Suk is an Assistant Professor of Applied Statistics at Teachers College, Columbia University. She is also a member of the Data Science Institute (DSI) at Columbia University. Dr. Suk specializes in the intersection of data science and causal inference in education, and her research program focuses on five key areas: (i) developing robust machine learning for causal inference in multilevel data, (ii) designing data-driven policy learning for personalized education, (iii) leveraging process data to advance causal inference and psychometric methods, (iv) evaluating algorithmic and test fairness in educational settings, and (v) utilizing generative models for method evaluation. She was recently honored with two early-career awards in her field—the 2026 American Educational Research Association (AERA) Division D Early-Career Award and the 2026 National Council on Measurement in Education (NCME) Alicia Cascallar Award—as well as the Outstanding Reviewer Award for the Journal of Educational and Behavioral Statistics (JEBS). Dr. Suk's research projects have been supported by the National Science Foundation (NSF), AERA, and the NAEd/Spencer Foundation. Her publications appear in Psychometrika, JEBS, and Multivariate Behavioral Research, among many others. Prior to joining Teachers College, she was an Assistant Professor in the School of Data Science at the University of Virginia. She completed her PhD in Quantitative Methods in the Department of Educational Psychology and her MS in Statistics, both at the University of Wisconsin-Madison, after earning her bachelor's and master's degrees from Seoul National University.