Designing Assessment to Support Learning: A New Approach to Test Construction and Analysis

Jimmy de la Torre

Assessments can be used not only to ascertain status of learning but also to further learning. However, current focus on accountability has overemphasized the use of assessments that lack diagnostic values. In contrast, assessments based on cognitive diagnosis models yield inferences that are richer and can facilitate learning. This proposal seeks to extend the applications of cognitive diagnosis modeling by introducing a general cognitively-based approach to test construction and analysis. Although the proposal focuses on multiple-choice assessments, minor modifications will allow application of the method to constructed response data. The first component of the approach prescribes how distractors in multiple-choice items, which can contain more information than the correct options, should be constructed. The second component describes how responses to these items should be analyzed using a proposed cognitive diagnosis model for multiple-choice data. Prior to investigation of its practical implications, several methodological questions about the proposed approach need to be answered. In addition to simulation studies, the project also seeks to develop and analyze a fourth grade cognitively-based multiple-choice assessment to gain a better understanding of the approach. The findings will be the basis for future practical applications such as studying how cognitively-based assessments can affect teaching practices and classroom learning.

About Jimmy de la Torre

Jimmy de la Torre holds a PhD in quantitative psychology from the University of Illinois at Urbana-Champaign. He is an assistant professor of Educational Psychology at the Graduate School of Education at Rutgers University. His primary research interests are in the field of psychological and educational measurement, with a specific emphasis on item response theory and latent variable modeling. As a researcher, his general objective is to develop and apply more appropriate methodologies using the most recent psychometric and computational developments to address pressing practical concerns in testing and measurement. He has been exploring ways of utilizing assessments to improve classroom instruction and learning through the use of cognitive diagnosis models which are developed specifically to identify the presence or absence of fine-grained skills. In addition to his current focus, which is the application of cognitive diagnosis in mathematics, he is also interested in collaborating with researchers from other fields to investigate how this framework can be extended and applied to other domains.