This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Kane, M. T. Articles by Brennan, R. L. Search for Related Content Social Bookmarking                         What's this?

Agreement Coefficients as Indices of Dependability for Domain-Referenced Tests

National League for Nursing

Robert L. Brennan

American College Testing Program

A large number of seemingly diverse coefficients have been proposed as indices of dependability, or reliability, for domain-referenced and/or mastery tests. In this paper it is shown that most of these indices are special cases of two generalized indices of agreement—one that is corrected for chance and one that is not. The special cases of these two in dices are determined by assumptions about the na ture of the agreement function or, equivalently, the nature of the loss function for the testing proce dure. For example, indices discussed by Huynh (1976), Subkoviak (1976), and Swaminathan,

Hambleton, and Algina (1974) employ a threshold agreement, or loss, function; whereas, indices dis cussed by Brennan and Kane (1977a, 1977b) and Livingston (1972a) employ a squared-error loss function. Since all of these indices are discussed within a single general framework, the differences among them in their assumptions, properties, and uses can be exhibited clearly. For purposes of com parison, norm-referenced generalizability coeffi cients are also developed and discussed within this general framework.

Applied Psychological Measurement, Vol. 4, No. 1, 105-126 (1980)
DOI: 10.1177/014662168000400111


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				R. R. Meijer and K. Sijtsma Methodology Review: Evaluating Person Fit Applied Psychological Measurement, June 1, 2001; 25(2): 107 - 135. [Abstract] [PDF]

				M.-n. F. Li and S. Olejnik The Power of Rasch Person-Fit Statistics in Detecting Unusual Response Patterns Applied Psychological Measurement, September 1, 1997; 21(3): 215 - 231. [Abstract] [PDF]

				J. J. Norcini, P. L. Stillman, A. I. Sutnick, M. B. Regan, H. L. Haley, R. G. Williams, and M. Friedman Scoring and Standard Setting with Standardized Patients Eval Health Prof, September 1, 1993; 16(3): 322 - 332. [Abstract] [PDF]

				J. J. Norcini, E. W. Hancock, G. D. Webster, L. J. Grosso, and J. A. Shea A Criterion-Referenced Examination of Physician Competence Eval Health Prof, March 1, 1988; 11(1): 98 - 112. [Abstract] [PDF]

				A. Nussbaum Multivariate Generalizability Theory in Educational Measurement: An Empirical Study Applied Psychological Measurement, April 1, 1984; 8(2): 219 - 230. [Abstract]

				M. Kane and J. Wilson Errors of Measurement and Standard Setting in Mastery Testing Applied Psychological Measurement, January 1, 1984; 8(1): 107 - 115. [Abstract]

				R. E. Traub and G. L. Rowley Reliability of Test Scores and Decisions Applied Psychological Measurement, October 1, 1980; 4(4): 517 - 545. [Abstract]

				R. L. Brennan and R. E. Lockwood A Comparison of the Nedelsky and Angoff Cutting Score Procedures Using Generalizability Theory Applied Psychological Measurement, April 1, 1980; 4(2): 219 - 240. [Abstract]