| Sign In to gain access to subscriptions and/or personal tools. |
Adjustments for Rater Effects in Performance AssessmentAmerican College Testing
American College Testing
American College Testing
Alternative methods to correct for rater leniency/stringency effects (i.e., rater bias) in per formance ratings were investigated. Rater bias effects are of concern when candidates are evaluated by different raters. The three correction methods evaluated were ordinary least squares (OLS), weighted least squares (WLS), and imputation of the missing data (IMPUTE). In addition, the usual procedure of averaging the observed ratings was investigated. Data were simulated from an essentially
Key Words: Index terms: EM algorithm, incomplete data • incomplete rating designs • least squares adjustments • performance assessment • rater calibration.
Applied Psychological Measurement, Vol. 15, No. 4,
409-421 (1991) This article has been cited by other articles:
|
|||||||||||||||
 |
|
|
 |
|
Sign In to gain access to subscriptions and/or personal tools.
-equivalent measure ment model, with true scores and error scores nor mally distributed. The variables manipulated in the simulations were method of correction (OLS, WLS, IMPUTE, averaging the observed ratings), amount of missing data (50% missing, 75% missing), rater bias (low, high), and number of examinees or can didates (N = 50, N = 100). The accuracy of the methods in estimating true scores was assessed based on the square root of the average squared difference between the estimated and known true scores. The three correction methods consistently outperformed the procedure of averaging the observed ratings. IMPUTE was superior to the least squares methods. 
