This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Shepard, L. Search for Related Content Social Bookmarking                         What's this?

Standard Setting Issues and Methods

University of Colorado

Previous methodological reviews and the contro versy regarding the adequacy of standard-setting technology are summarized. The judgmental nature of all standard-setting methods is examined, and the debate about whether fallible standards are better than none is recast in the context of three different test uses: pupil diagnosis, pupil certification (for high school graduation or professional licensure), and program evaluation. Exemplary standard-set ting methods are reviewed, representing the follow ing major approaches: (1) judgments of test con tent ; (2) judgments about mastery-nonmastery groups; (3) norms and passing rates; (4) empirical methods for discovering standards; and (5) empiri cal methods for adjusting cutoff scores, given a standard on an external criterion measure. Stan dards based on the performance of judged mastery groups (the Contrasting Groups method) and cer tain uses of normative data are likened to Known Groups validation. Recommendations are made for selecting standard-setting techniques depending on test use, including pupil diagnosis, pupil certifica tion, and program evaluation. Future research on standard setting is discussed in the context of im proving practical aspects of judgmental methods.

Applied Psychological Measurement, Vol. 4, No. 4, 447-467 (1980)
DOI: 10.1177/014662168000400403


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				G. Lee and D. M. Lewis A Generalizability Theory Approach to Standard Error Estimates for Bookmark Standard Settings Educational and Psychological Measurement, August 1, 2008; 68(4): 603 - 620. [Abstract] [PDF]

				Ping Yin and J. Sconing Estimating Standard Errors of Cut Scores for Item Rating and Mapmark Procedures: A Generalizability Theory Approach Educational and Psychological Measurement, February 1, 2008; 68(1): 25 - 41. [Abstract] [PDF]

				S. N. Beretvas Comparison of Bookmark Difficulty Locations Under Different Item Response Models Applied Psychological Measurement, January 1, 2004; 28(1): 25 - 47. [Abstract] [PDF]

				G. M. Hurtz and M. A. Auerbach A Meta-Analysis of the Effects of Modifications to the Angoff Method on Cutoff Scores and Judgment Consensus Educational and Psychological Measurement, August 1, 2003; 63(4): 584 - 601. [Abstract] [PDF]

				R. Janssen, F. Tuerlinckx, M. Meulders, and P. De Boeck A Hierarchical IRT Model for Criterion-Referenced Measurement Journal of Educational and Behavioral Statistics, January 1, 2000; 25(3): 285 - 306. [Abstract] [PDF]

				G. M. Hurtz and N. R. Hertz How Many Raters Should be Used for Establishing Cutoff Scores with the Angoff Method? a Generalizability Theory Study Educational and Psychological Measurement, December 1, 1999; 59(6): 885 - 897. [Abstract] [PDF]

				K. T. Taube The Incorporation of Empirical Item Difficulty Data into the Angoff Standard-Setting Procedure Eval Health Prof, December 1, 1997; 20(4): 479 - 498. [Abstract] [PDF]

				M. Kane Validating the Performance Standards Associated With Passing Scores Review of Educational Research, January 1, 1994; 64(3): 425 - 461. [Abstract] [PDF]

				D. W. Chambers The Effects of Faculty Control of Learning on Student Effort Eval Health Prof, December 1, 1991; 14(4): 438 - 455. [Abstract] [PDF]

				M. L. Fehrmann, D. J. Woehr, and W. Arthur Jr. The Angoff Cutoff Score Method: The Impact of Frame-of-Reference Rater Training Educational and Psychological Measurement, December 1, 1991; 51(4): 857 - 872. [Abstract]

				E. H. Haertel Chapter 1: New Forms of Teacher Assessment Review of Research in Education, January 1, 1991; 17(1): 3 - 29. [PDF]

				J. Millman If at First You Don't Succeed Setting Passing Scores When More Than One Attempt Is Permitted Educational Researcher, August 1, 1989; 18(6): 5 - 9. [Abstract] [PDF]

				A. R. Fitzpatrick Social Influences in Standard Setting: The Effects of Social Interaction on Group Judgments Review of Educational Research, January 1, 1989; 59(3): 315 - 328. [Abstract] [PDF]

				R. A. Berk A Consumer's Guide to Setting Performance Standards on Criterion-Referenced Tests Review of Educational Research, January 1, 1986; 56(1): 137 - 172. [Abstract] [PDF]

				L. J. Gross Setting Cutoff Scores on Credentialing Examinations: A Refinement in the Nedelsky Procedure Eval Health Prof, December 1, 1985; 8(4): 469 - 493. [Abstract] [PDF]

				R. R. Reilly, D. L. Zink, and E. W. Israelski Comparison of Direct and Indirect Methods For Setting Minimum Passing Scores Applied Psychological Measurement, September 1, 1984; 8(4): 421 - 429. [Abstract]

				R. K. Hambleton and S. Powell A Framework for Viewing the Process of Standard Setting Eval Health Prof, March 1, 1983; 6(1): 3 - 24. [Abstract] [PDF]