Internal Validity Tutorial


myAU\|	Login

d. Repeated Testing

The prior measurement of the dependent variable may affect the results obtained from subsequent measurements. Suppose that the dependent variable is recorded twice for a group of subjects, once at Time A and later at Time B, and that the independent variable is introduced in the interim. If scores on the dependent measure differ at these two times, the discrepancy may be due to the independent variable or to the procedure involved in measuring the dependent variable at Time A.

Background Information

Example

Due to time constraints, the experiment was run over four consecutive days. On Day 1, children viewed the 20-minute cartoon (Control condition). On Day 2, the Generalization Probe was conducted. On Day 3, the children were exposed to the 20-minute interactive video (Experimental condition). Finally, on Day 4, a second Generalization Probe was conducted. The mean score for children on the first Generalization Probe was 1.2 and their mean score on the second Generalization Probe was 3.4. We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation.

Nonexample

Due to time constraints, the experiment was run over three consecutive days. The name of each child in the classes was written on a separate slip of paper. All the slips were put in a bowl and mixed up thoroughly. Students were assigned to the Experimental Group and to the Control Group alternately as their names were pulled out of the bowl one at a time. On Day 1, a Generalization Probe was conducted for all children. On Day 2, the children in the comparison groups were separated, with the Control Group children viewing the 20-minute cartoon and the Experimental Group children viewing the 20-minute interactive video. On Day 3, a second Generalization Probe was conducted. To see the results, click here (1). We conclude that the 20-minute interactive video improved the children's self-protection skills in a potential abduction situation.

Analysis

The first item is an example in which repeated testing could be a threat to internal validity. A reasonable assumption is that few, if any, of the children in the experiment had experienced a potential abduction situation before the study started. On Day 1, they find themselves in this situation, and then again two days later. This unlikely state of affairs may have led at least some of them to believe that the second Generalization Probe was not real, but rather a test, especially if the probe procedure and the confederates running it were exactly the same. As a result, these children may have done what the teacher wanted them to do. The improvement across the two Generalization Probes may be due to intervening exposure to the interactive video or to the effects of prior experience with the Generalization Probe procedure on the second Generalization Probe score.

In the second item, if the improvement observed for the Experimental Group was a function of multiple exposure to the probe procedure, then we would also expect to see similar improvement for the Control Group. Because we do not observe this, we can be more confident that the improved score for the Experimental Group was not the result of repeated testing.

d. Repeated Testing

Copyright © 1998 Athabasca University