|
Sign In to gain access to subscriptions and/or personal tools.
|
Applied Psychological Measurement, Vol. 32, No. 2,
138-155 (2008)
DOI: 10.1177/0146621607300421
Conditional Covariance-Based Subtest Selection for DIMTEST
Amy G. Froelich
Iowa State University, amyf{at}iastate.edu
Brian Habing
University of South Carolina
DIMTEST is a nonparametric hypothesis-testing procedure designed to test the assumptions of a unidimensional and locally independent item response theory model. Several previous Monte Carlo studies have found that using linear factor analysis to select the assessment subtest for DIMTEST results in a moderate to severe loss of power when the exam lacks simple structure, the ability and difficulty parameter distributions differ greatly, or the underlying model is noncompensatory. A new method of selecting the assessment subtest for DIMTEST, based on the conditional covariance dimensionality programs DETECT and HCA/ CCPROX, is presented. Simulation studies show that using DIMTEST with this new selection method has either similar or significantly higher power to detect multidimensionality than using linear factor analysis for subtest selection, while maintaining Type I error rates around the nominal level.
Key Words: Index terms: DIMTEST • item response theory • unidimensionality • local independence • conditional covariance • linear factor analysis • HCA/CCPROX • DETECT
References
- Ackerman, T. (1996). Graphical representation of multidimensional item response theory. Applied Psychological Measurement, 20, 311-329.[Abstract]
- Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 395-479). Menlo Park, CA: Addison-Wesley.
- Drasgow, F. (1987). A study of the measurement bias of two standardized psychological tests. Journal of Applied Psychology, 72, 19-30.[CrossRef][ISI]
- Froelich, A.G. (2006). A new bias correction method for the DIMTEST procedure. Manuscript submitted for publication.
- Habing, B., & Roussos, L.A. (2003). On the need for negative local item dependence. Psychometrika, 68, 435-452.[CrossRef][ISI]
- Hattie, J., Krakowski, K., Rogers, J., & Swaminathan, H. (1996). An assessment of Stout's index of essential dimensionality. Applied Psychological Measurement, 20, 1-14.[Abstract]
- Hattie, J.A. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139-164.[Abstract]
- Kim, H.R. (1994). New techniques for the dimensionality assessment of standardized test data. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Department of Statistics.
- Lord, F.M. (1968). An analysis of the verbal scholastic aptitude test using Birnbaum's three-parameter logistic model. Educational and Psychological Measurement, 28, 989-1020.[CrossRef][ISI]
- Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
- McDonald, R.P. (1967). Nonlinear factor analysis. Psychometric Monographs, 15.
- Mislevy, R.J. (1986). Recent developments in the factor analysis of categorical variables. Journal of Educational Statistics, 11, 3-31.[CrossRef][ISI]
- Mislevy, R.J., & Bock, R.D. (1984). Item operating characteristics of the Armed Services Vocational Aptitude Battery (ASVAB), Form 8A (Technical Report N00014-83-C-0283). Washington, DC: Office of Naval Research.
- Nandakumar, R., & Stout, W. (1993). Refinements of Stout's procedure for assessing unidimensionality. Journal of Educational Statistics, 18, 41-68.[CrossRef][ISI]
- Reckase, M.D. (1997). A linear logistic model for dichotomous item response data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 271-286). New York: Springer.
- Roussos, L., Stout, W., & Marden, J. (1998). Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35, 1-30.[CrossRef][ISI]
- Seraphine, A.E. (2000). The performance of DIMTEST when latent trait and item difficulty distributions differ. Applied Psychological Measurement, 42, 82-94.
- Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589-617.[CrossRef][ISI]
- Stout, W., Froelich, A.G., & Gao, F. (2001). Using resampling to produce an improved DIMTEST procedure. In A. Boomsma, M. A. J. van Dujin, & T. A. B. Snijders (Eds.), Essays on item response theory (pp. 357-375). New York: Springer.
- Stout, W., Habing, B., Douglas, J., Kim, H.R., Roussos, L., & Zhang J. (1996). Conditional covariance-based nonparametric multidimensionality assessment. Applied Psychological Measurement, 19, 331-354.[Medline]
[Order article via Infotrieve]
- Sympson, J.B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 computerized adaptive testing conference (pp. 82-98). Minneapolis: University of Minnesota, Department of Psychology, Computerized Adaptive Testing Laboratory.
- van der Linden, W. J., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer.
- Zhang, J., & Stout, W. (1999a). Conditional covariance structure of generalized compensatory multidimensional items. Psychometrika, 64, 129-152.[CrossRef][ISI]
- Zhang, J., & Stout, W. (1999b). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika, 64, 213-249.[CrossRef][ISI]
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati What's this?
|
