Applied Psychological Measurement

 

Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Free Access - Register Here

Click here to sign up for SAGE Journal Email Alerts today!

Sign In to gain access to subscriptions and/or personal tools.
This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Google Scholar
Right arrow Articles by Emons, W. H. M.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
Applied Psychological Measurement, Vol. 32, No. 3, 224-247 (2008)
DOI: 10.1177/0146621607302479

Nonparametric Person-Fit Analysis of Polytomous Item Scores

Wilco H. M. Emons

Tilburg University, w.h.m.emons{at}uvt.nl

Person-fit methods are used to uncover atypical test performance as reflected in the pattern of scores on individual items in a test. Unlike parametric person-fit statistics, nonparametric person-fit statistics do not require fitting a parametric test theory model. This study investigates the effectiveness of generalizations of nonparametric person-fit statistics to polytomous item response data. A simulation study using varying test and item characteristics shows that a simple count of the Guttman errors is effective in detecting serious person misfit. The simulation study further shows that in most conditions a simple nonparametric person-fit statistic is as effective as a commonly used parametric person-fit statistic in detecting deviant item score vectors. An empirical example illustrates the use of the nonparametric person-fit statistics in real data.

Key Words: aberrant response behavior • nonparametric item response theory • person-fit analysis • person misfit • polytomous items

References

  • Bachman, J.G., & Malley, P.M. (1984). Yea-saying, nay-saying, and going to extremes: Black-white differences in response styles. Public Opinion Quarterly, 48, 491-509.[Abstract]
  • Birenbaum, M., & Nassar, F. (1994). On the relationship between test anxiety and test performance. Measurement and Evaluation in Counseling and Development, 27, 293-301.[ISI]
  • Cavalini, P.M. (1992). It's an ill wind that bring no goods. Studies on odour annoyance and the dispersion of odorant concentrations from industries. Unpublished doctoral dissertation, University of Groningen, Netherlands.
  • Chernyshenko, O.S., Stark, S., Chan, K., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523-562.[CrossRef][ISI]
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
  • Costa, P.T., & McCrae, R.R. (1992). The NEO Personality Inventory and NEO Five Factor Inventory professional manual. Odessa, FL: Psychological Assessment Resources.
  • Dagohoy, A.V.T. (2005). Person fit for tests with polytomous responses. Unpublished doctoral dissertation. Enschede, Netherlands: University of Twente.
  • Drasgow, F., Levine, M.V., & Williams, E.A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67-68.
  • Efron, B., & Tibshirani, R.J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
  • Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
  • Emons, W.H.M. (2003). Investigating the local fit of item-score vectors. In H. Yanai, A. Okada, K. Shigemasu, Y. Kano, & J. J. Meulman (Eds.), New developments in psychometrics (pp. 289-296). Tokyo: Springer.
  • Emons, W.H.M., Meijer, R.R., & Sijtsma, K. (2002). Comparing simulated and theoretical sampling distributions of the U3 person-fit statistic. Applied Psychological Measurement, 26, 88-108.[Abstract/Free Full Text]
  • Emons, W.H.M., Sijtsma, K., & Meijer, R.R. (2005). Global, local, and graphical person-fit analysis using person response functions. Psychological Methods, 10, 101-119.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Hamilton, D.L. (1968). Personality attributes associated with extreme response style. Psychological Bulletin, 69, 192-203.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Hemker, B.T., Sijtsma, K., & Molenaar, I.W. (1995). Selection of unidimensional scales from a multidimensional item bank in the polytomous Mokken IRT model. Applied Psychological Measurement, 19, 337-352.[Abstract]
  • Hemker, B.T., Sijtsma, K., Molenaar, I.W., & Junker, B.W. (1997). Stochastic ordering using the latent trait and the sum score in polytomous IRT models. Psychometrika, 62, 331-347.[CrossRef][ISI]
  • Johnson, T.R. (2004). On the use of heterogeneous thresholds ordinal regression models to account for individual differences in response style. Psychometrika, 68, 563-583.[CrossRef][ISI]
  • Karabatsos, G. (2003). Comparing the aberrant response detection performance of thirty-six person-fit statistics. Applied Measurement in Education, 16, 277-298.[CrossRef][ISI]
  • Meijer, R.R. (2003). Diagnosing item score patterns on a test using IRT based person-fit statistics. Psychological Methods, 8, 72-87.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Meijer, R.R., & Baneke, J. (2004). Analyzing psychopathology items: A case for nonparametric item response theory modeling. Psychological Methods, 9, 354-367.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Meijer, R.R., Molenaar, I.W., & Sijtsma, K. (1994). Influence of test and person characteristics on nonparametric appropriateness measurement. Applied Psychological Measurement, 18, 111-120.[Abstract/Free Full Text]
  • Meijer, R.R., & Sijtsma, K. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135.[Abstract/Free Full Text]
  • Molenaar, I.W. (1982). Mokken scaling revisited. Kwantitatieve Methoden, 3(8), 145-164.
  • Molenaar, I.W. (1991). A weighted Loevinger H-coefficient extending Mokken scaling to multicategory items. Kwantitatieve Methoden, 12(37), 97-117.
  • Molenaar, I.W. (1997). Nonparametric models for polytomous responses. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 369-380). New York: Springer.
  • Molenaar, I.W., & Hoijtink, H. (1990). The many null distributions of person-fit indices. Psychometrika, 55, 75-106.[CrossRef][ISI]
  • Molenaar, I.W., & Sijtsma, K. (2000). MSP5 for Windows. User's manual [Computer manual]. Groningen, Netherlands: ProGAMMA.
  • Paulhus, D.L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17-59). San Diego, CA: Academic Press.
  • Ramsay, J.O. (2000). Testgraf. A program for the graphical analysis of multiple choice test and questionnaire data [Computer software]. Montreal, Canada: Department of Psychology, McGill University.
  • Reise, S.P., & Widaman, K.F. (1999). Assessing the fit of measurement models at the individual level: A comparison of item response theory and covariance structure approaches. Psychological Methods, 4, 3-21.[CrossRef][ISI]
  • Reise, S.P., Widaman, K.F., & Puch, R.H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Rennie, L.J. (1982). Research note: Detecting a response set to Likert-style attitude items with the rating model. Education Research and Perspectives, 9, 114-118.
  • Rivas, T., Bersabé, R., & Berrocal, C. (2005). Application of the double monotonicity model to polytomous items: Scalability of the Beck depression items on subjects with eating disorders. European Journal of Psychological Assessment, 21, 1-10.[CrossRef][ISI]
  • Rossi, P.E., Gilula, Z., & Allenby, G.M. (2001). Overcoming scale usage heterogeneity: A Bayesian hierarchical approach. Journal of the American Statistical Association, 96, 20-31.[CrossRef][ISI]
  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 17.
  • Samejima, F. (1997). The graded response model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 85-100). New York: Springer.
  • Sijtsma, K., & Meijer, R.R. (2001). The person response function as a tool in person-fit research. Psychometrika, 66, 191-208.[CrossRef][ISI]
  • Sijtsma, K., & Molenaar, I.W. (2002). Introduction to nonparametric item response theory. Thousand Oaks, CA: Sage.
  • Sijtsma, K., & Van der Ark, L.A. (2003). Investigation and treatment of missing item scores in test and questionnaire data. Multivariate Behavioral Research, 38, 505-528.[CrossRef][ISI]
  • Steinberg, L., & Thissen, D. (1996). Uses of item response theory and the testlet concept in the measurement of psychopathology. Psychological Methods, 1, 81-97.[CrossRef][ISI]
  • Thissen, D. (1991). MULTILOG user's guide. Multiple categorical item analysis and test scoring using item response theory [Computer manual]. Chicago: Scientific Software.
  • Van der Ark, L.A. (2001). Relationships and properties of polytomous item response theory models. Applied Psychological Measurement, 25, 273-282.[Abstract/Free Full Text]
  • Van der Flier, H. (1980). Vergelijkbaarheid van individuele testprestaties [Comparability of individual test performance]. Lisse, Netherlands: Swets & Zeitlinger.
  • Van der Flier, H. (1982). Deviant response patterns and comparability of test scores. Journal of Cross-Cultural Psychology, 13, 267-298.[Abstract]
  • Van Herk, H., Poortinga, Y.H., & Verhallen, T.M.M. (2004). Response styles in rating scales: Evidence of method bias in data from six EU countries. Journal of Cross Cultural Psychology, 35, 346-360.[Abstract]
  • Van Krimpen-Stoop, E.M.L. A., & Meijer, R.R. (2002). Detection of person misfit in computerized adaptive tests with polytomous items. Applied Psychological Measurement, 26, 164-180.[Abstract/Free Full Text]
  • Van Onna, M.J.H. (2003). Ordered latent class models in nonparametric item response theory. Unpublished doctoral dissertation, University of Groningen, Netherlands.
  • Zickar, M.J., & Drasgow, F. (1996). Detecting faking on a personality instrument using appropriateness measurement. Applied Psychological Measurement, 20, 71-88.[Abstract]
  • Zickar, M.J., Gibby, R.E., & Robie, C. (2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed model item response theory. Organizational Research Methods, 7, 168-190.[Abstract]

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?



This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Google Scholar
Right arrow Articles by Emons, W. H. M.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?