Mixture Computerized Adaptive Testing
ACCOMMODATING DIVERSITY IN MEASURING PERSON-CENTRED HEALTH OUTCOMESComputerized adaptive tests (CATs) use statistical algorithms to identify questions that are most informative based on an individual’s responses to questions that have already been administered. The advantage of using CATs is that they can minimize response burden by selectively administering those questions that are most likely to be relevant to an individual’s health status. With their application, the most informative measurement at a desired level of precision can be obtained with efficiency. Relative to conventional measurement approaches that require that every person responds to the same complete set of items, CATs can be shorter, uniquely targeted to an individual’s status, and more accurate.
Despite these important advantages, there are challenges regarding the use of CATs in diverse and heterogeneous populations. Research has shown that people in diverse populations with different backgrounds and life-experiences may not be consistent in how they interpret and respond to questions about their health and QOL. People’s interpretations and responses may be influenced by differences in their age, gender, bodyweight, ethnicity, or other factors. Biases in person-centred outcomes measurement will result when such inconsistencies are ignored.
Inconsistencies in how people interpret and respond to measurement items could result in over-estimation or under-estimation of scores in heterogeneous populations. Mixture-CAT can be used to address such inconsistencies by taking sources of heterogeneity into account. This is achieved by applying heterogeneity-adjusted item selection and scoring algorithms to existing item banks, based on latent variable mixture models. The procedures for doing so are described in the following publications listed below.(1,2)
The demo provides an example applying a mixture-CAT to an existing item bank for measuring, which is part of the CAT-5D-QOL.(3) The items and scoring algorithms match those described in the first publication.
Demonstration
The demo begins with a few background questions about sources of heterogeneity associated with differences in how people interpret and respond to questions about their pain.
Subsequently, a mixture-CAT is administered accommodating for these sources of heterogeneity.
The resulting scores are shown to demonstrate how the accuracy of score improves following the administration of each question.
In addition, scores are compared to those that would have been obtained when using a conventional CAT that does not adjust for heterogeneity.
References
1. Sawatzky, R., Ratner, P. A., Kopec, J. A., Wu, A. D., & Zumbo, B. D. (2016). The accuracy of computerized adaptive testing in heterogeneous populations: A mixture item-response theory analysis. PLOS One, 11(3), e0150563. doi: 10.1371/journal.pone.0150563
2. Sawatzky, R., Ratner, P. A., Kopec, J. A., & Zumbo, B. D. (2012). Latent variable mixture models: A promising approach for the validation of patient reported outcomes. Quality of Life Research, 21(4), 637-650. doi: 10.1007/s11136-011-9976-6
3. Kopec, J. A., Badii, M., McKenna, M., Lima, V. D., Sayre, E. C., & Dvorak, M. (2008). Computerized adaptive testing in back pain: Validation of the CAT-5D-QOL. Spine, 33(12), 1384-1390. doi: 10.1097/BRS.0b013e3181732a3b
Presentation
Exploring the potential of a mixture-computerized adaptive test for use in heterogeneous populations
PROMIS®: Global Advances in Methodology and Clinical Science. Dublin, Ireland. October, 2018.