This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Fleiss, J. L. Search for Related Content Social Bookmarking                         What's this?

Balanced Incomplete Block Designs for Inter-Rater Reliability Studies

Columbia University and New York State Psychiatric Institute

Occasionally, an inter-rater reliability study must be designed so that each subject is rated by fewer than all the participating raters. If there is interest in comparing the raters' mean levels of rating, and if it is desired that each mean be estimated with the same precision, then a balanced incomplete block design for the reliability study is indicated. Methods for executing the design and for analyzing the resulting data are presented, using data from an actual study for illustration.

Applied Psychological Measurement, Vol. 5, No. 1, 105-112 (1981)
DOI: 10.1177/014662168100500115


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				C. A. Cameron and J. Hutchison Telephone-mediated communication effects on young children's oral and written narratives First Language, November 1, 2009; 29(4): 347 - 371. [Abstract] [PDF]

				S. Seider "Bad Things Could Happen": How Fear Impedes Social Responsibility in Privileged Adolescents Journal of Adolescent Research, November 1, 2008; 23(6): 647 - 666. [Abstract] [PDF]

				J. Roberts, G. E. Martin, L. Moskowitz, A. A. Harris, J. Foreman, and L. Nelson Discourse Skills of Boys With Fragile X Syndrome in Comparison to Boys With Down Syndrome J Speech Lang Hear Res, April 1, 2007; 50(2): 475 - 492. [Abstract] [Full Text] [PDF]

				C. W. T. Chiu and E. W. Wolfe A Method for Analyzing Sparse Data Matrices in the Generalizability Theory Framework Applied Psychological Measurement, September 1, 2002; 26(3): 321 - 338. [Abstract] [PDF]

				W. T. Hoyt and J. N. Melby Dependability of Measurement in Counseling Psychology: An Introduction to Generalizability Theory The Counseling Psychologist, May 1, 1999; 27(3): 325 - 352. [Abstract] [PDF]

				G. Dunn Review papers : Design and analysis of reliability studies Statistical Methods in Medical Research, August 1, 1992; 1(2): 123 - 157. [Abstract] [PDF]

				M. R. Raymond, L. C. Webb, and W. M. Houston Correcting Performance-Rating Errors in Oral Examinations Eval Health Prof, March 1, 1991; 14(1): 100 - 122. [Abstract] [PDF]

				H. I. Braun Understanding Scoring Reliability: Experiments in Calibrating Essay Readers Journal of Educational and Behavioral Statistics, January 1, 1988; 13(1): 1 - 18. [Abstract] [PDF]