[S03] Theory choice

Whether in scientific research or in everyday life, we often need to choose between alternative explanations or theories. Here are six criteria we can use to evaluate them and help us decide which to accept.

1. Consistency with observations

What are the facts or observations we are trying to explain? Are they incompatible with any of the theories? If so, this will be a good reason to reject them, unless there are reasons to think that some of the observations are not reliable.

2. Predictive power

A scientific theory ought to help us make predictions and explain our observations. If a hypothesis generates no testable prediction, it fails the minimal requirement for a scientific hypothesis.

When we evaluate the predictive power of a theory, we consier both the quantity and the quality of the predictions. How many predictions can the theory make? How accurate and precise are they? Does the theory make predictions across a wide range of phenomena?

3. Mechanism

In general, we want theories that can explain the connections between events by revealing the underlying causal mechanisms. This can help us generate more predictions to test the theory and make other discoveries.

4. Fruitfulness

This is about whether a theory helps us make surprising or unexpected predictions which turn out to be correct, and whether the theory helps us detect and explain connections which we would not have noticed otherwise.

5. Simplicity

A simple theory is (roughly) one with fewer assumptions, and which posits less entities than its competitors. Many scientists believe strongly that we should search for simple theories if feasible.

6. Coherence

A theory should be internally coherent in the sense that it is logically consistent. If not, there is something wrong with the theory as it stands, and so there is a need to revise the theory to come up with a better version.

The other aspect of coherence is that we should look for theories that fit together with other well-confirmed facts and scientific theories. Widely accepted theories are already well-confirmed, so if a hypothesis is incompatible with existing science, the default response should be that the hypothesis is mistaken. An extraordinary claim incompatible with scientific knowledge should require very strong evidence before it can be accepted.

---

One piece of writing very relevant to the topic under discussion comes from the famous scientist and writer Carl Sagan. In one of his books he proposed what he calls “A Baloney Detection Kit,” a set of tools useful for scientific and everyday reasoning. Here they are:

1. Wherever possible there must be independent confirmation of the “facts.”
2. Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
3. Arguments from authority carry little weight — “authorities” have made mistakes in the past. They will do so again in the future. Perhaps a better way to say it is that in science there are no authorities; at most, there are experts.
4. Spin more than one hypothesis. If there’s something to be explained, think of all the different ways in which it could be explained. Then think of tests by which you might systematically disprove each of the alternatives.
5. Try not to get overly attached to a hypothesis just because it’s yours. It’s only a way station in the pursuit of knowledge. Ask yourself why you like the idea. Compare it fairly with the alternatives. See if you can find reasons for rejecting it. If you don’t, others will.
6. If whatever it is you’re explaining has some measure, some numerical quantity attached to it, you’ll be much better able to discriminate among competing hypotheses. What is vague and qualitative is open to many explanations.
7. If there’s a chain of argument, every link in the chain must work (including the premise) — not just most of them.
8. Occam’s Razor. This convenient rule-of-thumb urges us when faced with two hypotheses that explain the data equally well to choose the simpler.
9. Always ask whether the hypothesis can be, at least in principle, falsified ... You must be able to check assertions out. Inveterate skeptics must be given the chance to follow your reasoning, to duplicate your experiments and see if they get the same result.