Cognitive Diagnosis Models for Learning: Measuring Fluency Using Response Time and Response Accuracy

Shiyu Wang

The recent "Every Student Succeed Act"? encourages schools to use "innovative assessment to provide feedback about students"? mastery level of grade-level content standards"?. Mastery of a skill requires the ability to complete the task with fluency. Computerized assessment emerges as an important tool for measuring fluency due to its capacity to provide rich information on both response accuracy and speed of performance. Albeit these advantages, computerized assessment data are hardly utilized to infer fluency due to the lack of psychometric models. This proposed project aims to address this limitation by developing a family of psychometric models within the Cognitive Diagnosis Model framework that use response accuracy and response time to measure skill fluency both in static and dynamic case. An exploratory data analysis on the results from a computerized learning program will be first conducted. A family of confirmatory models based on the exploratory findings will be proposed. These models will be evaluated through both Monte Carlo simulation studies and a real data application. The developed statistical tools will help teachers and educators to monitor students' skill fluency. They will also help evaluate learning interventions and school curriculum by assessing whether students achieve the learning goals set in a curriculum.

About Shiyu Wang

Dr. Shiyu Wang is an Assistant Professor in Quantitative Methodology Program of Department of Educational Psychology at University of Georgia. She received her Ph.D. in Statistics from University of Illinois at Urbana-Champaign. Her research program contributes to personalized assessment and learning through the following three perspectives:1) developing innovative adaptive testing that can provide efficient individualized assessments and an examinee-friendly testing environment; 2) establishing statistical foundations for a family of restricted latent class models to provide guidelines for model estimation and selection; 3) developing novel dynamic psychometric models that can measure and predict students? learning outcomes based on various assessment data, including product data, students? responses, and process data (response time and learning time) to facilitate the development of adaptive learning. Dr. Wang received 2019 Early Career Researcher Award from the International Association for Computerized Adaptive Testing and her research findings have been published in top notch journals such as Journal of Educational Measurement, Applied Psychological Measurement, Psychometrika and British Journal of Mathematical and Statistical Psychology.