By now you will probably agree that high credibility is far from easy to obtain. However, that does not mean there are no (or almost no) credible studies; it is just that the credible ones are almost always limited. They are far more specialized than we would like them to be and burdened with more ifs, whens, and presumablys than we would prefer. There is not much sense in complaining about this situation; it is simply an unavoidable result of the complexity of the world in which we live and (as far as technology is concerned) that we have created. And it would not even be much of a problem if we were patient enough to be happy with the things we do find out.

And here is what we believe to be the real problem: although engineers and scientists understand a lot about complexity, have respect for it and for the effort it often implies, and are still capable of awe in that regard, our society and culture as a whole do not. We are surrounded by so many spectacular things and events that we come to feel that small news is no news. A finding that requires an intricate 50-word clause to summarize it without distortion is unlikely to receive our attention.

The general media act accordingly. In order to capture attention, they ignore, inflate, or distort the findings of an empirical study, often beyond recognition. Scientists often are not helpful either and write so-called abstracts that merely announce results rather than summarizing them. In either case, the burden is on the critical reader to take a closer look. You will have to dig out a report about the study, digest it, decide its credibility, and take home what is both credible and relevant for you. Your qualifications as a software engineer mean you are able to do this, and progress in software engineering depends on many engineers actually exercising this capability on a regular basis. This book represents a wonderful opportunity to do just that.

One thing that observational studies have shown is that expert software engineers do gather evidence when it matters to them, at a level of credibility pitched for their purpose. That is, if they’re designing digital filters for professional audio, they may well do controlled experiments on outputs to determine whether the software produces the desired sound qualities for human hearing. If they’re in a battle with marketing over whether customers who currently use analog controls are ready to handle a virtual interface, they may well send a researcher into the field to discover what parameters the customers are controlling, how their existing systems function, and how the customers think about their tasks. If they’re designing frontends for safety-critical systems, they may well do user studies to determine which of several design alternatives best fit the human context of use. If they’re designing backends for safety-critical systems, they may well use formal methods to establish correct implementation of the specified requirements. If they’re optimizing scheduling algorithms, they may well run benchmarks against industrial inputs.

So rather than dismissing evidence of various forms, we should embrace it, and think hard and critically about it.