This Article Full Text (OnlineFirst[PDF]) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Google Scholar Articles by Hurtz, G. M. Articles by Jones, C. N. Social Bookmarking                   What's this?

Conversion of Proportion-Correct Standard-Setting Judgments to Cutoff Scores on the Item Response Theory Scale

^* To whom correspondence should be addressed. E-mail: ghurtz{at}csus.edu.

	Abstract

This study compares the efficacy of different strategies for translating item-level, proportion-correct standard-setting judgments into a -metric test cutoff score for use with item response theory (IRT) scoring, using Monte Carlo methods. Simulated Angoff-type ratings, consisting of 1,000 independent 75 Item x13 Rater matrices, were generated at five points along the continuum, at three levels of rater fit to the item characteristics curves, yielding 14,625,000 ratings as the basis of the analyses. These simulated proportion-correct ratings were converted to the IRT scale using test-level and item-level methods explicated by Kane (1987). Kane’s optimally weighted, item-level conversion method initially produced anomalous results; however, it was discovered that imposing a restriction on the weights avoided these anomalies and rendered the optimally weighted method the most statistically efficient. Six areas for future research are outlined for advancing the integration of these classical standard-setting ratings into IRT methodology.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?