Multidimensional Analysis of Student Growth Using Item Response Theory

Chun Wang

Recently, the federal government instituted a program entitled “Race to the Top”, in which schools are encouraged to “build data systems that measure student growth and success”. To help the government improve measuring student ability, psychometricians have proposed sophisticated models that presumably pinpoint students’ strength and weakness on relevant content areas. However, the currently available models mainly focus on data collected at a single time point, and no systematic efforts have extended these models to measure individual change (across multiple time points). This study focuses on the development, evaluation, and application of longitudinal extensions of multidimensional higher-order item response theory models. These longitudinal hierarchical IRT (L-HIRT) models will be developed to measure individual change (with a focus on aptitude/academic change) over time. The flexibility of the new L-HIRT models allows incorporating test designs that include multidimensional content areas, multiple time point measurements, and multiple subpopulations. Simulation studies will be conducted to explore the recovery of all model parameters and to demonstrate the applicability of the new models in reliably measuring growth. Multivariate hypothesis testing will also be proposed to evaluate the significance of individual change. Finally, the NELS (National Educational Longitudinal Studies) dataset will be used to illustrate specific L-HIRT model applications. I hope that this study will provide useful statistical tools for reliably reporting and evaluating individual growth on a general, overall trait, and across several, more specific content domains.

About Chun Wang

Dr. Chun Wang is an assistant professor at the University of Minnesota in the Department of Psychology and specializes in quantitative and psychometric methods. She received her PhD in Quantitative Psychology from the University of Illinois at Urbana-Champaign in 2012. Her research interests include multidimensional and multilevel item response theory, computerized adaptive testing, cognitive diagnosis modeling, test security, and semi-parametric modeling for item response time analysis. Since 2012, she has received two innovative Research & Development grants from CTB/McGraw-Hill to develop (1) advanced psychometric models for measuring individual change/growth in longitudinal studies; (2) sophisticated statistical models for differentiating various test-taking behaviors , including solution behavior, cheating behavior, and rapid guessing behavior, by way of integrating information from both response pattern and response time pattern. Dr. Wang has received numerous awards including the 2011 Nancy Hirschberg Memorial Award, 2012 Jeffrey Tanaka Memorial Award, 2013 JEBS Best Reviewer Award, NCME 2013 Alicia Cascallar Award, and NCME 2014 Jason Millman Promising Measurement Scholar Award.