This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Norcini, J. Articles by Grosso, L. Search for Related Content Social Bookmarking                   What's this?

The Effect of Numbers of Experts and Common Items on Cutting Score Equivalents Based on Expert Judgment

American Board of Internal Medicine

Judy Shea

American Board of Internal Medicine

Louis Grosso

American Board of Internal Medicine

The effect of different numbers of experts and common items on the scaling of cutting scores derived by experts' judgments was investigated. Four test forms were created from each of two examinations; each form from the first examina tion shared a block of items with one form from the second examination. Small groups of experts set standards on each using a modification of Angoff's (1971) method. Cutting score equivalents were estimated for the matched forms using dif ferent group sizes and numbers of common items; they were compared with cutting score equivalents based on score equating. Results showed that a reduction in error is associated with using more experts or having more items in common between the two forms. For 25 or more common items and five or more judges, the error was about one item on a 100-item test. More than five experts or 25 common items made only a very small difference in error.

Key Words: Index terms: cutting scores • equating, expert judgment • standard setting.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?