|
Sign In to gain access to subscriptions and/or personal tools.
|
Internal Invalidity in Pretest-Posttest Self-Report Evaluations and a Re-evaluation of Retrospective Pretests
George S. Howard
University of Houston
Kenneth M. Ralph
Lancaster County Office of Mental Health and Mental Retardation
Nancy A. Gulanick
University of Houston
Scott E. Maxwell
University of Houston
Don W. Nance
Wichita State University
Sterling K. Gerber
Eastern Washington State College
True experimental designs (Designs 4, 5, and 6 of Campbell & Stanley, 1963) are thought to provide internally valid results. This paper describes five studies involving the evaluation of various treat ment interventions and identifies a source of inter nal invalidity when self-report measures are used in a Pretest-Posttest manner. An alternative approach (Retrospective Pretest-Posttest design) to measuring change is suggested, and data comparing its ac curacy with the traditional Pretest-Posttest design in measuring treatment effects is presented. Finally, the implications of these findings for evaluation re search using self-report instruments and the strengths and limitations of retrospective measures are discussed.
Applied Psychological Measurement, Vol. 3, No. 1,
1-23 (1979)
DOI: 10.1177/014662167900300101
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
D. Moore and C. A. Tananis
Measuring Change in a Short-Term Educational Program Using a Retrospective Pretest Design
American Journal of Evaluation,
June 1, 2009;
30(2):
189 - 202.
[Abstract]
[PDF]
|
|
|
|
|
|
|
P. J. Taylor, D. F. Russ-Eft, and H. Taylor
Gilding the Outcome by Tarnishing the Past: Inflationary Biases in Retrospective Pretests
American Journal of Evaluation,
March 1, 2009;
30(1):
31 - 43.
[Abstract]
[PDF]
|
|
|
|
|
|
|
G. Holden, K. Barker, G. Rosenberg, and P. Onghena
The Evaluation Self-Efficacy Scale for Assessing Progress Toward CSWE Accreditation Related Objectives: A Replication
Research on Social Work Practice,
January 1, 2008;
18(1):
42 - 46.
[Abstract]
[PDF]
|
|
|
|
|
|
|
G. Holden, K. Barker, G. Rosenberg, and P. Onghena
Assessing Progress Toward Accreditation Related Objectives: Evidence Regarding the Use of Self-Efficacy as an Outcome in the Advanced Concentration Research Curriculum
Research on Social Work Practice,
July 1, 2007;
17(4):
456 - 465.
[Abstract]
[PDF]
|
|
|
|
|
|
|
H. Razmjou, A. Yee, M. Ford, and J. A. Finkelstein
Response Shift in Outcome Assessment in Patients Undergoing Total Knee Arthroplasty
J. Bone Joint Surg. Am.,
December 1, 2006;
88(12):
2590 - 2595.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
T. C. M. Lam and P. Bengo
A Comparison of Three Retrospective Self-reporting Methods of Measuring Change in Instructional Practice
American Journal of Evaluation,
March 1, 2003;
24(1):
65 - 80.
[Abstract]
[PDF]
|
|
|
|
|
|
|
C. C. Pratt, W. M. McGuigan, and A. R. Katzev
Measuring Program Outcomes: Using Retrospective Pretest Methodology
American Journal of Evaluation,
September 1, 2000;
21(3):
341 - 349.
[Abstract]
[PDF]
|
|
|
|
|
|
|
G. Holden, L. Cuzzi, S. Rutter, P. Chernack, and G. Rosenberg
The Hospital Social Work Self-Efficacy Scale: A Replication
Research on Social Work Practice,
October 1, 1997;
7(4):
490 - 499.
[Abstract]
[PDF]
|
|
|
|
|
|
|
J. Francis-Smythe and P. M. Smith
The Psychological Impact of Assessment in a Development Center
Human Relations,
February 1, 1997;
50(2):
149 - 167.
[Abstract]
[PDF]
|
|
|
|
|
|
|
F. R. Rohs and C. A. Langone
Increased Accuracy in Measuring Leadership Impacts
Journal of Leadership and Organizational Studies,
January 1, 1997;
4(1):
150 - 158.
[Abstract]
[PDF]
|
|
|
|
|
|
|
G. Holden, L. Cuzzi, S. Rutter, G. Rosenberg, and P. Chernack
The Hospital Social Work Self-Efficacy Scale: Initial Development
Research on Social Work Practice,
July 1, 1996;
6(3):
353 - 365.
[Abstract]
[PDF]
|
|
|
|
|
|
|
C. A. Dickenson, C. D. Fisher, J. B. Shaw, and G. N. Southey
Teaching HRM and Managerial Skills with the 'Living Case'
Asia Pacific Journal of Human Resources,
March 1, 1996;
33(3):
39 - 52.
[Abstract]
[PDF]
|
|
|
|
|
|
|
E. A. R. Robinson and H. J. Doueck
Implications of the Pre/Post/Then Design for Evaluating Social Group Work
Research on Social Work Practice,
April 1, 1994;
4(2):
224 - 239.
[Abstract]
[PDF]
|
|
|
|
|
|
|
K. M. Skeff, G. A. Stratos, and M. R. Bergen
Evaluation of a Medical Faculty Development Program: A Comparison of Traditional Pre/Post and Retrospective Pre/Post Self-Assessment Ratings
Eval Health Prof,
September 1, 1992;
15(3):
350 - 366.
[Abstract]
[PDF]
|
|
|
|
|
|
|
L. S. Aiken and S. G. West
Invalidity of True Experiments: Self-Report Pretest Biases
Eval Rev,
August 1, 1990;
14(4):
374 - 390.
[Abstract]
[PDF]
|
|
|
|
|
|
|
W. A. Randolph and D. F. Elloy
How Can OD Consultants and Researchers Assess Gamma Change? A Comparison of Two Analytical Procedures
Journal of Management,
December 1, 1989;
15(4):
633 - 648.
[Abstract]
[PDF]
|
|
|
|
|
|
|
D. A. Payne
How I Learned to Love "The Standards"
American Journal of Evaluation,
November 1, 1988;
9(4):
37 - 44.
[PDF]
|
|
|
|
|
|
|
D. A. Payne
How I Learned to Love "The Standards"
American Journal of Evaluation,
August 1, 1988;
9(3):
35 - 42.
[PDF]
|
|
|
|
|
|
|
K. N. Wexley and T. T. Baldwin
Management Development
Journal of Management,
June 1, 1986;
12(2):
277 - 294.
[Abstract]
[PDF]
|
|
|
|
|
|
|
L. M. Collins, J. W. Graham, W. B. Hansen, and C. A. Johnson
Agreement Between Retrospective Accounts of Substance Use and Earlier Reported Substance Use
Applied Psychological Measurement,
September 1, 1985;
9(3):
301 - 309.
[Abstract]
|
|
|
|
|
|
|
J. H. Bray, S. E. Maxwell, and G. S. Howard
Methods of Analysis with Response-Shift BIAS
Educational and Psychological Measurement,
December 1, 1984;
44(4):
781 - 804.
[Abstract]
|
|
|
|
|
|
|
N. C. Roberts and J. I. Porras
Progress in Organization Development Research
Group Organization Management,
March 1, 1982;
7(1):
91 - 116.
[Abstract]
|
|
|
|
|
|
|
S. E. Maxwell and G. S. Howard
Change Scores--Necessarily Anathema?
Educational and Psychological Measurement,
October 1, 1981;
41(3):
747 - 756.
[Abstract]
|
|
|
|
|
|
|
G. S. Howard, J. Millham, S. Slaten, and L. O'Donnell
Influence of Subject Response Style Effects on Retrospective Measures
Applied Psychological Measurement,
January 1, 1981;
5(1):
89 - 100.
[Abstract]
|
|
|
|
|
|
|
G. S. Howard
A New Look at the Criterion Problem from the Perspective of Gelso's Bubble Hypothesis
The Counseling Psychologist,
December 1, 1980;
8(4):
63 - 69.
|
|
|
|
|
|
|
G. S. Howard
Response-Shift Bias: A Problem in Evaluating Interventions with Pre/Post Self-Reports
Eval Rev,
February 1, 1980;
4(1):
93 - 106.
[Abstract]
[PDF]
|
|
|
|
|
|
|
G. S. Howard, P. R. Dailey, and N. A. Gulanick
The Feasibility of Informed Pretests in Attenuating Response-Shift Bias
Applied Psychological Measurement,
October 1, 1979;
3(4):
481 - 494.
[Abstract]
|
|
|
|
|
