This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Google Scholar Articles by Wang, W.-C. Social Bookmarking                   What's this?

A Critique of Raju and Oshima's Prophecy Formulas for Assessing the Reliability of Item Response Theory-Based Ability Estimates

National Chung Cheng University, Taiwan, psywcw{at}ccu.edu.tw

Raju and Oshima (2005) proposed two prophecy formulas based on item response theory in order to predict the reliability of ability estimates for a test after change in its length. The first prophecy formula is equivalent to the classical Spearman-Brown prophecy formula. The second prophecy formula is misleading because of an underlying false assumption. It can yield negative reliability estimates, especially if the reliability of ability estimates for the old test is low and the new test is shorter. This article identifies the fallacy of the second prophecy formula and demonstrates the scope of its bias in predicting test reliability.

Key Words: Index terms: Spearman-Brown prophecy formula • item response theory • test reliability • measurement error • classical test theory

References

• Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397-479). Reading, MA: Addison-Wesley.
• Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 3, 296-322.
• Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
• Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Nijhoff.
• Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
• Lord, F.M. (1983). Unbiased estimators of ability parameters, of their variance, and of their parallel-forms reliability. Psychometrika, 48, 233-245.[CrossRef][ISI]
• Mislevy, R.J., & Bock, R.D. (1990). BILOG3: Item analysis and test scoring with binary logistic models. Mooresville, IN: Scientific Software.
• Raju, N.S., & Oshima, T.C. (2005). Two prophecy formulas for assessing the reliability of item response theory-based ability estimates. Educational and Psychological Measurement, 65, 361-375.[Abstract]
• Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danmarks Paedogogische Institut.
• Samejima, F. (1994). Estimation of reliability coefficients using the test information function and its modifications. Applied Psychological Measurement, 18, 229-244.[Abstract/Free Full Text]
• Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 3, 271-295.
• Wright, B.D., & Masters, G.N. (1982). Rating scale analysis. Chicago: MESA Press.

CiteULike

Connotea

Del.icio.us

Digg

Reddit

Technorati

This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Google Scholar Articles by Wang, W.-C. Social Bookmarking                   What's this?