Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Click here for more information on Research and Evaluation in Education and Psychology, 3e

CiteULike is a free service for managing and discovering scholarly references - click here to get started.

Sign In to gain access to subscriptions and/or personal tools.
Applied Psychological Measurement
This Article
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
0146621607301495v1
32/5/385    most recent
Right arrow References
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via Google Scholar
Right arrow Citing Articles via Scopus
Google Scholar
Right arrow Articles by Hurtz, G. M.
Right arrow Articles by Jones, C. N.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?

Conversion of Proportion-Correct Standard-Setting Judgments to Cutoff Scores on the Item Response Theory {theta} Scale

Gregory M. Hurtz

California State University, Sacramento, ghurtz{at}csus.edu

J. Patrick Jones

Excelsior College

Christian N. Jones

Taleo Corporation

This study compares the efficacy of different strategies for translating item-level, proportion-correct standard-setting judgments into a y-metric test cutoff score for use with item response theory (IRT) scoring, using Monte Carlo methods. Simulated Angoff-type ratings, consisting of 1,000 independent 75 Item x 13 Rater matrices, were generated at five points along the y continuum, at three levels of rater fit to the item characteristics curves, yielding 14,625,000 ratings as the basis of the analyses. These simulated proportion-correct ratings were converted to the IRT y scale using test-level and item-level methods explicated by Kane (1987). Kane's optimally weighted, item-level conversion method initially produced anomalous results; however, it was discovered that imposing a restriction on the weights avoided these anomalies and rendered the optimally weighted method the most statistically efficient. Six areas for future research are outlined for advancing the integration of these classical standard-setting ratings into IRT methodology.

Key Words: standard-setting • Angoff method • Angoff method • item response theory • cutoff scores • model fit • Monte Carlo • simulation

This version was published on July 1, 2008

Applied Psychological Measurement, Vol. 32, No. 5, 385-406 (2008)
DOI: 10.1177/0146621607301495
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?