Foreword by Walter Bright
There’s a line in a science fiction novel I read long ago that says a scientist would fearlessly peer into the gates of hell if he thought it would further knowledge in his field. In one sentence, it captures the essence of what it means to be a scientist. This joy in discovery, this need to know, is readily apparent in the videos and writings of physicist Richard Feynman, and his enthusiasm is infectious and enthralling.
Although I am not a scientist, I understand their motivation. Mine is that of an engineer—the joy of creation, of building something out of nothing. One of my favorite books is a chronicle of the step-by-step process the Wright brothers went through to solve the problems of flight one by one, The Wright Brothers as Engineers by Wald, and how they poured all that knowledge into creating a flying machine.
My early interests were summed up in the opening pages of Rocket Manual for Amateurs by Brinley with the phrase “thrilled and fascinated by things that burn and explode,” later matured into wanting to build things that went faster and higher.
But building powerful machines is an expensive proposition. And then I discovered computers. The marvelous and seductive thing about computers is the ease with which things can be built. You don’t need a billion-dollar fab plant, a machine shop, or even a screwdriver. With just an inexpensive computer, you can create worlds.
So I started creating imaginary worlds on the computer. The first was the game Empire, Wargame of the Century. The computers of the day were too underpowered to run it properly, so I became interested in how to optimize the performance of programs. This led to studying the compilers that generated the code and naturally to the hubris of “I can write a better compiler than that.” Enamored with C, I gravitated toward implementing a C compiler. That wasn’t too hard, taking a couple of years part-time. Then I discovered Bjarne Stroustrup’s C++ language, and I thought that I could add those extensions to the C compiler in a couple of months (!).
Over a decade later, I was still working on it. In the process of implementing it, I became very familiar with every detail of the language. Supporting a large user base meant a lot of experience in how other people perceived the language, what worked, and what didn’t. I’m not able to use something without thinking of ways to improve the design. In 1999, I decided to put this into practice. It started out as the Mars programming language, but my colleagues called it D first as a joke, but the name caught on and the D programming language was born.
D is ten years old as of this writing and has produced its second major incarnation, sometimes called D2. In that time D has expanded from one man toiling over a keyboard to a worldwide community of developers working on all facets of the language and supporting an ecosystem of libraries and tools.
The language itself (which is the focus of this book) has grown from modest beginnings to a very powerful language adept at solving programming problems from many angles. To the best of my knowledge, D offers an unprecedentedly adroit integration of several powerful programming paradigms: imperative, object-oriented, functional, and meta.
At first blush, it would appear that such a language could not be simple. And indeed, D is not a simple language. But I’d argue that is the wrong way to view a language. A more useful view is, what do programming solutions in that language look like? Are D programs complicated and obtuse, or simple and elegant?
A colleague of mine who has extensive experience in a corporate environment observed that an IDE (Integrated Development Environment) was an essential tool for programming because with one click a hundred lines of boilerplate code could be generated. An IDE is not as essential a tool for D, because instead of relying on wizard-based boilerplate generation, D obviates the boilerplate itself by using introspection and generational capabilities. The programmer doesn’t have to see that boilerplate. The inherent complexity of the program is taken care of by the language, rather than an IDE.
For example, suppose one wanted to do OOP (object-oriented programming) using a simpler language that has no particular support for the paradigm. It can be done, but it’s just awful and rarely worthwhile. But when a more complex language supports OOP directly, then writing OOP programs becomes simple and elegant. The language is more complicated, but the user code is simpler. This is worthwhile progress.
The ability to write user code for a wide variety of tasks in a simple and elegant manner pretty much requires a language that supports multiple programming paradigms. Properly written code should just look beautiful on the page, and beautiful code oddly enough tends to be correct code. I’m not sure why that relationship holds, but it tends to be true. It’s the same way an airplane that looks good tends to fly well, too. Therefore, language features that enable algorithms to be expressed in a beautiful way are probably good things.
Simplicity and elegance in writing code, however, are not the only metrics that characterize a good programming language. These days, programs are rapidly increasing in size with no conceivable end in sight. With such size, it becomes less and less practical to rely on convention and programming expertise to ensure the code is correct, and more and more worthwhile to rely on machine-checkable guarantees. To that end, D sports a variety of strategies that the programmer can employ to make such guarantees. These include contracts, memory safety, various function attributes, immutability, hijack protection, scope guards, purity, unit tests, and thread data isolation.
No, we haven’t overlooked performance! Despite many predictions that performance is no longer relevant, despite computers running a thousand times faster than when I wrote my first compiler, there never seems to be any shortage of demand for faster programs. D is a systems programming language. What does that mean? In one sense, it means that one can write an operating system in D, as well as device drivers and application code. In a more technical sense, it means that D programs have access to all the capabilities of the machine. This means you can use pointers, do pointer aliasing and pointer arithmetic, bypass the type system, and even write code directly in assembly language. There is nothing completely sealed off from a D programmer’s access. For example, the implementation of D’s garbage collector is entirely written in D.
But wait! How can that be? How can a language offer both soundness guarantees and arbitrary pointer manipulation? The answer is that the kinds of guarantees are based on the language constructs used. For example, function attributes and type constructors can be used to state guarantees enforcible at compile time. Contracts and invariants specify guarantees to be enforced at runtime.
Most of D’s features have appeared in other languages in one form or another. Any particular one doesn’t make the case for a language. But the combination is more than the sum of the parts, and D’s combination makes for a satisfying language that has elegant and straightforward means to solve an unusually wide variety of programming problems.
Andrei Alexandrescu is famous for his unconventional programming ideas becoming the new mainstream (see his seminal book Modern C++ Design). Andrei joined the D programming language design team in 2006. He’s brought with him a sound theoretical grounding in programming, coupled with an endless stream of innovative solutions to programming design problems. Much of the shape of D2 is due to his contributions, and in many ways this book has co-evolved with D. One thing you’ll happily discover in his writing about D is the why of the design choices, rather than just a dry recitation of facts. Knowing why a language is the way it is makes it much easier and faster to understand and get up to speed.
Andrei goes on to illustrate the whys by using D to solve many fundamental programming problems. Thus he shows not only how D works, but why it works, and how to use it.
I hope you’ll have as much fun programming in D as I’ve had working to bring it to life. A palpable excitement about the language seeps out of the pages of Andrei’s book. I think you’ll find it exciting!
Walter Bright
January 2010