Repeating a test checks that the result is consistent. A fair design checks that the comparison is valid. A result can have agreeing repeats and still come from an unfair test.
This lesson is part of the set on evaluating a science claim from a supplied dataset. The fair-test idea is taught from the start in identifying variables and fair comparisons.
What are the two questions?
- Did the repeats agree? If yes, the result is reliable, which means you would get a similar result again.
- Was only one thing changed? If yes, the design is fair, which means the difference can be blamed on that one change.
Reliable and fair are separate. Answer both for every dataset.
Worked example: agreeing repeats, unfair design
An original test compares plant food with water only. Each group was measured in three repeats, each repeat using 4 new plants, and the average height after 14 days is shown in cm.
| Repeat | With plant food | Water only |
|---|---|---|
| 1 | 18 | 15 |
| 2 | 18 | 15 |
| 3 | 19 | 16 |
The test notes add, “The food plants sat on the sunny window ledge. The water plants sat on a shelf at the back of the room.”
Question 1: did the repeats agree? Yes. The food group gave 18, 18 and 19 cm, and the water group gave 15, 15 and 16 cm. The repeats are close, so the result is reliable.
Question 2: was only one thing changed? No. Two things differ: the plant food and the amount of light. The food plants got more sunlight.
Conclusion. The repeats agree, but the design is not fair, so the 3 cm difference cannot be blamed on plant food. The extra light could have caused it, or both together.
Correction. Put all plants on the same ledge, in alternating positions, and change only the plant food.
Why do the repeats not fix the problem?
The unfair light affected every repeat in the same way. Each repeat gave the food plants more light, so each repeat leaned in the same direction. Agreement was guaranteed, and it says nothing about the cause.
The mistake that costs marks
The slip is to write “the test is fair because it was repeated three times”.
| Wrong | Right |
|---|---|
| “It was repeated, so it is fair” | “It was repeated, so it is reliable, but the light differed, so it is not fair” |
| “The results agree, so plant food works” | “The results agree, but the design does not isolate plant food” |
| “Do more repeats to make it fair” | “Change the design so light is the same, then repeat” |
The repair is to give the reliability verdict and the fairness verdict separately.
Check yourself
A fictional test compares two detergents on the same type of stain. Detergent M was used with water at 50°C and Detergent N with water at 20°C.
Each was tested in 3 repeats, and the stain was fully removed in M in every repeat and partly removed in N in every repeat. Is the result reliable? Is the test fair? What should change?
Answer
Reliable: yes, because the repeats agree. M removed the stain every time and N removed it only partly every time.
Fair: no. Two things differ, the detergent and the water temperature (50°C against 20°C), so the result cannot be blamed on the detergent.
Change: use water at the same temperature for both detergents, then repeat.
What to study next
The last lesson, rewriting an overconfident conclusion so it matches the evidence, puts the four checks into the final sentence. The graph evidence and fair-comparison lab offers designs to inspect.
To practise with a teacher, see online one-to-one Science tuition.