This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Zimmerman, D. W. Articles by Zumbo, B. D. Search for Related Content Social Bookmarking                         What's this?

Reliability of Measurement and Power of Significance Tests Based on Differences

Carleton University

Richard H. Williams

University of Miami

Bruno D. Zumbo

University of Ottawa

The power of significance tests based on differ ence scores is indirectly influenced by the reliability of the measures from which differences are obtained. Reliability depends on the relative magnitude of true score and error score variance, but statistical power is a function of the absolute magnitude of these components. Explicit power calculations reaffirm the paradox put forward by Overall & Woodward (1975, 1976)—that significance tests of differences can be powerful even if the reliability of the difference scores is 0. This anomaly arises because power is a function of observed score variance but is not a function of reliability unless either true score variance or error score variance is constant. Provided that sample size, significance level, directionality, and the alternative hypothesis associated with a significance test remain the same, power always increases when population variance decreases, independently of reliability.

Key Words: Index terms: difference scores • error of measurement • power, significance tests • t test • test reliability • true scores.

Applied Psychological Measurement, Vol. 17, No. 1, 1-9 (1993)
DOI: 10.1177/014662169301700101


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				D. K. Oller and H. L. Ramsdell A Weighted Reliability Measure for Phonetic Transcription J Speech Lang Hear Res, December 1, 2006; 49(6): 1391 - 1411. [Abstract] [Full Text] [PDF]

				J. T. Austin, K. A. Boyle, and J. C. Lualhati Statistical Conclusion Validity for Organizational Science Researchers: A Review Organizational Research Methods, April 1, 1998; 1(2): 164 - 208. [Abstract]

				R. H. Williams and D. W. Zimmerman Are Simple Gain Scores Obsolete? Applied Psychological Measurement, March 1, 1996; 20(1): 59 - 69. [Abstract]

				L. G. Humphreys Further Comments on Reliability and Power of Significance Tests Applied Psychological Measurement, March 1, 1993; 17(1): 11 - 14. [Abstract]

				D. W. Zimmerman, R. H. Williams, and B. D. Zumbo Reliability, Power, Functions, and Relations: A Reply to Humphreys Applied Psychological Measurement, March 1, 1993; 17(1): 15 - 16.