Use of the IRT Model to Validate Test Items from a Technology Assisted Health Coaching Program
- 1 Department of Biostatistics and Epidemiology, UNT Health Science Center, Fort Worth, TX, USA
- 2 Department of Biostatistics and Epidemiology, UNT Health Science Center, Fort Worth, TX, USA
- 3 Department of Health Behavior and Health Systems, UNT Health Science Center, Fort Worth, TX, USA
- 4 Department of Health Behavior and Health Systems, UNT Health Science Center, Fort Worth, TX, USA
- 5 Department of Health Behavior and Health Systems, UNT Health Science Center, Fort Worth, TX, USA
Abstract
Item Response Theory (IRT) models have been extensively used in the field of education to identify link between a response to a test item and underlying latent capability of the test taker. We demonstrate the benefit of using IRT model to analyze health data using data from M. chat program such that statisticians can use the method in lieu of traditional methods including Cronbach’s alpha, discriminant analysis and factor analysis. M. chat is a technology based health coaching program and the baseline survey from the participants in the program include s response in different but correlated domains of diet, social habits, leisure practices, mental health, substance abuse, self-sufficiency and medication adherence. We analyzed baseline data from 416 subjects using IRT models. Our results indicated that responses pertaining to alcohol and substance abuse were the most discriminating items with an average discrimination estimate of approximately 4.99 whereas the least discriminating items were the diet habits, with an average estimate of - 0.476.
- An, X.M. and Yung, Y.F. (2014) Item Response Theory: What It Is and How You Can Use the IRT Procedure to Apply It. SAS Institute, North Carolina.
- Sheu, C., Chen, C., Su, Y.H. and Wang W. (2005) Using SAS PROC NLMIXED to Fit Item Response Theory Models. Behavior Research Methods, 37, 202-218. https://doi.org/10.3758/BF03192688
- Harris, D. (1989) Comparison of 1-, 2-, and 3-Parameter IRT Models. Educational Measurement: Issues and Practice, 8, 35-41. https://doi.org/10.1111/j.1745-3992.1989.tb00313.x
- Chang, C.H. and Reeve, B.B. (2005) Item Response Theory and Its Applications to Patient-Reported Outcomes Measurement. Evaluation & the Health Professions, 28, 264-282. https://doi.org/10.1177/0163278705278275
- Hambleton, R.K. (2000) Emergence of Item Response Modeling in Instrument Development and Data Analysis. Medical Care, 39, S60-S65.
- Drasgow, F. and Hulin, C.L. (1990) Item Response Theory. Handbook of Industrial and Organizational Psychology, 1, 577-636.
- Embretson, S.E. and Reise, S.P. (2013) Item Response Theory. Psychology Press, NY.
- Reeve, B.B., Hays, R.D., Bjorner, J.B., Cook, K.F., Crane, P.K., Teresi, J.A., et al. (2007) PROMIS Cooperative Group. Medical Care, 45, S22-S33. https://doi.org/10.1097/01.mlr.0000250483.85507.04
- Fries, J.F., Bruce, B. and Cella, D. (2005) The Promise of PROMIS: Using Item Response Theory to Improve Assessment of Patient-Reported Outcomes. Clinical and Experimental Rheumatology, 23, S53-S57.
- Hartman, C.A., Gelhorn, H., Crowley, T.J., Sakai, J.T., Stallings, M., Young, S.E. and Hopfer, C.J. (2008) Item Response Theory Analysis of DSM-IV Cannabis Abuse and Dependence Criteria in Adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 165-173. https://doi.org/10.1097/chi.0b013e31815cd9f2
- Grimby, G., Andrén, E., Holmgren, E., Wright, B., Linacre, J.M. and Sundh, V. (1996) Structure of a Combination of Functional Independence Measure and Instrumental Activity Measure Items in Community-Living Persons: A Study of Individuals with Cerebral Palsy and Spina Bifida. Archives of Physical Medicine and Rehabilitation, 77, 90131-90138. https://doi.org/10.1016/S0003-9993(96)90131-8
- Hays, R.D., Morales, L.S. and Reise, S.P. (2000) Item Response Theory and Health Outcomes Measurement in the 21st Century. Medical Care, 39, II28-II42.
- (2013) A Simple Guide to the Item Response Theory (IRT) and Rasch Modeling. http://www.creative-wisdom.com/computer/sas/IRT.pdf