Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Click here for more information on Research and Evaluation in Education and Psychology, 3e

CiteULike is a free service for managing and discovering scholarly references - click here to get started.

Sign In to gain access to subscriptions and/or personal tools.
Applied Psychological Measurement
This Article
Right arrow Full Text (PDF)
Right arrow References
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Right arrow Citing Articles via Scopus
Google Scholar
Right arrow Articles by Overall, J. E.
Right arrow Articles by Magee, K. N.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?

Estimating Individual Rater Reliabilities

John E. Overall

University of Texas Medical School

Kevin N. Magee

University of Texas Medical School

Rating scales have no inherent reliability that is independent of the observers who use them. The often reported interrater reliability is an average of perhaps quite different individual rater reliabilities. It is possible to separate out the individual rater reliabilities given a number of independent raters who observe the same sample of ratees. Under cer tain assumptions, an external measure can replace one of the raters, and individual reliabilities of two independent raters can be estimated. In a somewhat similar fashion, estimates of treatment effects present in ratings by two independent raters can provide the external frame of reference against which dif ferences in their individual reliabilities can be evaluated. Models for estimating individual rater reliabilities are provided for use in selecting, evalu ating, and training participants in clinical research.

Key Words: Index terms: attenuation • correlation • individual raters • interrater reliability • multiple raters • rater reli ability • rating scales • reliability of ratings • significance.

Applied Psychological Measurement, Vol. 16, No. 1, 77-85 (1992)
DOI: 10.1177/014662169201600109
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?


This article has been cited by other articles:


Home page
 Assessment for Effective InterventionHome page
B. Garrett, E. Towles, H. Kleinert, and J. Kearns
Portfolios in Large-Scale Alternate Assessment Systems: Frameworks for Reliability
Assessment for Effective Intervention, January 1, 2003; 28(2): 17 - 27.
[Abstract] [PDF]

Home page
 Applied Psychological MeasurementHome page
M. Wilson and W.-c. Wang
Complex Composites: Issues That Arise in Combining Different Modes of Assessment
Applied Psychological Measurement, March 1, 1995; 19(1): 51 - 71.
[Abstract]