6.1 What is an Experiment?

To discuss experimental method we need a definition to set it apart from mere observation. Since the experimenter interferes with the world to obtain useful data, we could say that an experiment involves a strategic manipulation of a system to create an organized response, in order to answer a specific question. These three criteria are a good start, but are they sufficient? When putting some oranges on the scales at the fruit market to find out what we should pay for them, we are making a strategic manipulation of a system (the scales) to create an organized response (the readout), in order to answer a specific question (what should we pay). Most people would probably agree that this is no more an experiment than looking at the speedometer when driving your car. Setting up a system and obtaining information from it by passive observation is not experimentation. When Mr. Green counted his apples in Chapter 2 he was not experimenting. If he had tested different fertilizers to increase the yield of apples, he would have been conducting an experiment.

An experiment imposes a treatment on something. It exerts a stimulus on the system under study. As experimenters we are interested in the response to this stimulus, because we hope it will say something about how the system works. If we were interested in how the fruit scales worked and put different objects on them to see how the readout was affected, this would be an experiment – an active interaction with the system under study. If we were only interested in the weight of the oranges, weighing them cannot be considered to be an experiment. This is because the response we are interested in is not created by us. The oranges would weigh the same whether we weighed them or not.

The crucial difference is this: the passive observer investigates a pre-existing state (Figure 6.1). The experimenter changes the state and investigates the result of the change (Figure 6.2). Would you, for example, conduct an experiment if you weighed yourself on the bathroom scales every morning? No you would not, because you would be measuring something that was already there. On the other hand, weighing yourself regularly while testing a new diet is an experiment, because you are measuring a response to a stimulus that you created.

Figure 6.1 A passive observer investigates a pre-existing state without attempting to influence it.

Figure 6.2 An experimenter changes the state and investigates the resulting response.

But what is the relevance of this discussion? The topic of this book is experimental method. It would be pointless to discuss this method if it could not be distinguished from other methods. We need some defining characteristics before we proceed. Another important point is that understanding the potential of the experimental method makes it is easier to make good experiments; this aspect is discussed further in the next section. Many Ph.D. students are experimenting intuitively, collecting data without a clear question in mind and trying to extract interesting information from the data afterwards. This is very similar to passive observation. An effective experiment addresses a specific question and requires the development of a clear strategy.

As we concluded in Chapter 4, experimentation is a much more efficient way of obtaining knowledge than passive observation. By creating conditions that are relevant to a specific question we ensure that the information we obtain is relevant. Sometimes, scientists must develop new instruments and apparatus to measure things, but no matter how cleverly a measurement system is conceived or how complex it is, just measuring something does not turn an investigation into an experiment. Experimenters do not settle with just counting or measuring things. If we agree that an experiment must measure a response to a stimulus exerted by the experimenter, this means that there is no fundamental difference between measuring the gravitational constant, counting neutrinos in a neutrino observatory, mapping the cosmic background radiation with a satellite, counting the apples in your garden or measuring the length of a cucumber. They are all forms of passive observation.

It may seem like an unfortunate consequence of our definition that it excludes some classical “experiments” in the history of science, such as the ingenious “Cavendish experiment”, which was the first to yield an accurate value of the universal gravitational constant. Cavendish basically devised a very clever balance that measured the gravitational force between lead balls without being disturbed by the earth's gravity. Saying that this is not an experiment is not to belittle the ingenuity and sheer beauty of his setup, nor to belittle its relevance to science. To define what an experiment is, a line must simply be drawn somewhere between experiment and passive observation.

The word “passive” is used here to contrast against the active, interfering nature of experimentation. It may be perceived as a charged word, so it should be stated that passive observation is also a very useful and important method in science. This is an undisputable fact, since there are many examples of great theories that have sprung from passive observation. For example, Edwin Hubble discovered that the degree of redshift observed in the spectra of galaxies increased with the galaxies’ distance from the earth. This helped him establish that the universe is expanding. The birds that Charles Darwin collected on the Galapagos Islands during his voyage on HMS Beagle played an important role in his development of the theory of evolution by natural selection. Alfred Wegener noticed that the continents of the earth seem to fit together like a jigsaw puzzle and suggested the hypothesis of continental drift. This was later developed into the modern theory of plate tectonics. All of these important discoveries were made through passive observation by attentive, shrewd observers. In many fields of study it is simply not possible to perform experiments, and in most other cases you must of course first notice something interesting before you are able to investigate it experimentally. Passive observation has contributed tremendously to the growth of scientific knowledge but it is not the topic of this book. The important point is that when we do have a choice between passive observation and active experimentation, we should choose experimentation because it is a more efficient way to learn how things work.

Many students at the beginning of their Ph.D. only have vague ideas about how to conduct experimental research. I once asked a student why he had chosen to manipulate a particular set of variables in his first experiment. He answered “well, you must change something, otherwise it is not an experiment”. After graduating he agreed that this statement, though basically true, was a bit naïve. Part of the reason for the confusion may be that school often gives us the false impression that experimentation is mainly about collecting data. During traditional laboratory classes we have used setups that were tested and tuned for us by our teachers, and followed predetermined steps to collect data. These activities have pedagogic value but lack an important aspect of real experiments: the development of a strategy to address a research question. Real experiments are preceded by a period of preparation where the idea for the experiment is developed and perfected. We will soon look at some illustrative examples of this, but first we will discuss the importance of how we formulate research questions and how we go about answering them.