[T14] Controlled trials

Module: Basic statistics

Quote of the page

I must be allowed to add some explanatory remarks to bring the subject home to reason—to that sluggish reason, which supinely takes opinions on trust, and obstinately supports them to spare itself the labour of thinking.

- Mary Wollstonecraft

Help us promote
critical thinking!

Popular pages

§1. Controlled trials

As we saw in the previous section, the fact that two things are correlated does not by itself show that one causes the other. To establish causal claims, statisticians use a method called the controlled trial. For example, suppose you wish to know whether a particular medicine is effective at curing baldness. What you wish to know is whether taking the medicine causes hair to grow. To establish this, you divide your test subjects into two groups, chosen at random (to avoid bias). Half the subjects (the treatment group) are given the medicine. The other half (the control group) are given a placebo, which looks just like the medicine but contains no active ingredients. You can then measure the hair growth in the two groups and compare the results. If the medicine is effective, then the difference between the two groups should be large enough that you can reject the null hypothesis that the medicine works no better than the placebo, at your chosen significance level.

The important point here is the random assignment of patients to the two groups. The only factor that differs between the groups in a systematic (as opposed to random) way is the treatment, so if there is a difference in hair growth, then either the medicine caused it, or it was the result of pure chance (sampling error). As long as the probability that the result was due to chance is small enough (smaller than the chosen significance level), we can conclude that the medicine was the cause.

Controlled trials are difficult to perform in many areas. For example, one cannot perform a controlled trial to test whether smoking causes cancer in humans, because one cannot take two groups at random and force one group to smoke for twenty years! Instead, the groups that are studied are self-selected--some people choose to smoke and some choose not to. There is always the possibility that those people who choose to smoke have some other factor in common that causes cancer. This makes it much more difficult to establish causation in this area. However, controlled trials on animals can be used, and these strongly support a causal link.

The trial for the baldness treatment described above is a blind trial. This means that the patients do not know whether they are in the treatment group or the control group--this is why the patients in the control group are given a placebo. Blind trials are used wherever possible. Why might this be important?
The importance of blind trials is to make sure that no factor varies systematically between the treatment group and the control group other than the treatment itself. If the patients know which group they are in, then this knowledge itself constitutes a systematic difference between the two groups. This might not seem like an important difference, but in fact simply believing that you are being treated can have a measurable effect for a wide range of medical conditions. Patients who are given a placebo to take often show a measurable improvement compared to patients who are given nothing at all. This is called the placebo effect.
A double blind trial is one in which neither the patient nor the doctor knows whether the patient is in the treatment group or the control group. The doctor doesn't know whether the tablets they give to the patient are genuine or placebos. Why might this be important?
Again, the importance here is to rule out any factors which might vary systematically between the treatment group and the control group. It has been shown that if the doctor knows whether the patient is in the treatment group or the control group, this knowledge can unconsciously alter their behaviour towards the patient and their perception of the patient's condition. In this case, if the doctor knows which group the patient is in, this might unconsciously influence their perception of slight changes in hair growth or of the severity of side effects. This makes the data provided by the doctor less reliable.
Reread the excerpt from the South China Morning Post in the previous question set. Would it be possible to conduct a controlled trial to determine whether superstitious beliefs cause neurosis, depression and low IQ?
No, a controlled trial is unlikely to be possible in this case. We can't assign people randomly to two groups and make one group believe in superstitions. Hence the two groups (superstitious and non-superstitious) are self-selecting, and this introduces the possibility of bias. It may be that people with a predisposition to neurosis, depression or low IQ are more likely to end up in the superstitious group than in the non-superstitious group. This is why the correlation reported in the newspaper article does not establish a causal link.

previous tutorial next tutorial