INTRODUCTION
A few years ago, I was riding on a ski lift with our Swedish exchange student. I asked her if she had thought about what she was going to do after high school. She said that she was considering engineering and had taken a programming class the previous year. I asked her what they taught. She replied, “Java.” I instinctively responded with “That’s too bad.”
Why did I say that? Took me a while to figure it out. It’s not that Java is a bad programming language; it’s actually pretty decent. I said it because of the way in which Java (and other languages) are typically used to teach programming today—without teaching anything about computers. If this strikes you as a bit odd, then this book is for you.
The Java programming language was invented by James Gosling, Mike Sheridan, and Patrick Naughton in the 1990s at Sun Microsystems. It was modeled in part after the C programming language, which was widely used at the time. C doesn’t include automatic management of memory, and memory management errors were a common headache at the time. Java eliminated that class of programming errors by design; it hid the underlying memory management from the programmer. That’s part of what makes it such a good programming language for beginners. But it takes much more than a good programming language to produce good programmers and programs. And it turned out that Java introduced a whole new class of harder-to-debug programming problems, including poor performance resulting from the hidden memory management system.
As you’ll see in this book, understanding memory is a key skill for programmers. When you’re learning to program, it’s easy to develop habits that become hard to break. Studies have shown that children who grew up playing at so-called “safe” playgrounds have a higher rate of injuries later in life than those who didn’t, presumably because they didn’t learn that falling hurts. Programming is an analogous situation. Safe programming environments make getting started less scary, but you also need to prepare for the outside world. This book helps you make that transition.
Why Good Programming is Important
To understand why it’s problematic to teach computer programming without also teaching about computers, first consider how ubiquitous computers have become. The price of computers has fallen so dramatically that using them is now the cheapest way to build many things. For example, using a computer to display an image of an old-fashioned analog clock on a car dashboard costs much less than a mechanical clock. This is a result of how computer chips are manufactured; they’re more or less printed. It’s no longer a big deal to stamp out a chip that contains billions of components. Note that I’m talking about the price of computers themselves, not the price of things that contain computers. In general, a computer chip today costs less than the packaging in which it’s shipped. Computer chips are available that cost pennies. There will likely come a time when it will be difficult to find anything that doesn’t contain a computer.
Lots of computers doing lots of things means lots of computer programs. Because computers are so ubiquitous, the field of computer programming is incredibly diverse. As in medicine, many programmers become specialists. You can specialize in areas such as vision, animation, web pages, phone apps, industrial control, medical devices, and more.
But the strange thing about computer programming is that unlike in medicine, in programming you can become a specialist without ever being a generalist. You probably wouldn’t want a heart surgeon who never learned anatomy, but the equivalent has become normal for many programmers today. Is this really a problem? In fact, there’s plenty of evidence that this isn’t working very well, with almost daily reports of security breaches and product recalls. There have been court cases in which people convicted of drunk driving by breathalyzer have won the right to have the breathalyzer code reviewed. It turned out that the code was full of bugs, which resulted in overturned convictions. Recently, a piece of antivirus software crashed a piece of medical equipment in the middle of a heart surgery. Lives were lost due to design issues in the Boeing 737 MAX airplane. The large number of incidents like these don’t inspire a lot of confidence.
Learning to Code is Only a Starting Place
Part of the reason for this state of affairs is that it’s not all that difficult to write computer programs that appear to work, or work much of the time. Let’s use the changes in music (not disco!) in the 1980s as an analogy. People used to have to develop a foundation in order to make music. This included learning music theory, composition, and how to play an instrument; ear training; and lots of practicing. Then the Musical Instrument Digital Interface (MIDI) standard, originally proposed by Ikutaro Kakehashi of Roland, came along, which let anyone make “music” from their computer without ever having to develop calluses. It’s my opinion that only a small percentage of computer-generated “music” is actually music; it’s mostly noise. Music is produced by actual musicians—who may or may not use MIDI to build on their foundation. Programming these days has become a lot like using MIDI. You no longer have to sweat much or spend years practicing or even learn theory in order to write programs. But that doesn’t mean these are good or reliable programs.
This situation is likely to get worse, at least in the United States. Wealthy people with vested interests, like those who own software companies, have been lobbying for legislation mandating that everybody learn to code in school. This sounds great in theory, but it’s not a great idea in practice because not everybody has the aptitude to become a good programmer. We don’t mandate that everybody learn to play football because we know that it’s not for everybody. The likely goal of this initiative is not to produce great programmers but rather to increase software company profits by flooding the market with large numbers of poor programmers, which will drive down wages. The people behind this push don’t care very much about code quality—they also push for legislation that limits their liability for defective products. Of course, you can program for fun just like you can play football for fun. Just don’t expect to be drafted for the Super Bowl.
In 2014, President Obama said that he had learned to code. He did drag a few things around in the excellent visual programming tool Blockly, and he even typed in one line of code in JavaScript (a programming language unrelated to Java, which was invented at Netscape, the predecessor to the Mozilla Foundation that maintains numerous software packages, including the Firefox web browser.) Now, do you think that he actually learned to code? Here’s a hint: if you do, you should probably work on honing your critical thinking skills in addition to reading this book. Sure, he may have learned a teensy bit about programming, but no, he didn’t learn to code. If he could learn to code in an hour, then it follows that coding is so trivial that there wouldn’t be a need to teach it in schools.
Importance of Low-Level Knowledge
An interesting and somewhat contrary view about how to teach programming was expressed in a blog post titled “How to Teach Computational Thinking” by Stephen Wolfram, the creator of Mathematica and the Wolfram language. Wolfram defines computational thinking as “formulating things with enough clarity, and in a systematic enough way, that one can tell a computer how to do them.” I completely agree with this definition. In fact, it’s in large part my motivation for writing this book.
But I strongly disagree with Wolfram’s position that those learning to program should develop computational thinking skills using powerful high-level tools, such as those that he’s developed, instead of learning the underlying foundational technologies. For example, it’s clear from the rising interest in statistics over calculus that “data wrangling” is a growing field. But what happens when people just feed giant piles of data into fancy programs that those same people don’t intimately understand?
One possibility is that they generate interesting-looking but meaningless or incorrect results. For example, a recent study (“Gene Name Errors Are Widespread in the Scientific Literature” by Mark Ziemann, Yotam Eren, and Assam El-Osta) showed that one-fifth of published genetics papers have errors due to improper spreadsheet usage. Just think of the kinds of errors and ramifications that more powerful tools in the hands of more people could produce! Getting it right is crucial when people’s lives are affected.
Understanding underlying technologies helps you develop a sense of what can go wrong. Knowing just high-level tools makes it easy to ask the wrong questions. It’s worth learning to use a hammer before graduating to a nail gun. Another reason for learning underlying systems and tools is that it gives you the power to build new tools, which is important because there will always be a need for tool builders, even if tool users are more common. Learning about computers so that the behavior of programs isn’t a mystery enables you to craft better code.
Who Should Read This Book?
This book is for people who want to become good programmers. What makes a good programmer? First and foremost, a good programmer has good critical thinking and analysis skills. To solve complex problems, a programmer needs the ability to evaluate whether or not programs actually solve the right problem correctly. This is more difficult than it sounds. It’s not uncommon for an experienced programmer to look at someone else’s program and snarkily comment, “Why, that’s a complex nonsolution to a simple nonproblem.”
You may be familiar with a classic fantasy trope of wizards acquiring power over things by learning their true names. And woe be to the wizard who forgets a detail. Good programmers are like these wizards who can hold the essence of things in their minds without dropping details.
Good programmers also have some degree of artistry, like skilled craftspeople. It’s not uncommon to find code that is completely incomprehensible, just like many English speakers are baffled by James Joyce’s novel Finnegans Wake. Good programmers write code that not only works but is also easy for others to understand and maintain.
Finally, good programmers need a deep understanding of how computers work. You can’t solve complex problems well using a shallow base of knowledge. This book is for people who are learning programming but are unsatisfied with the lack of depth. It’s also for people who are already programming but want more.
What Are Computers?
A common answer is that computers are appliances that people use for tasks such as checking email, shopping online, writing papers, organizing photos, and playing games. This definition is partly the result of sloppy terminology that became commonplace as consumer products began to incorporate computers. Another common answer is that computers are the brains that make our high-tech toys, such as cell phones and music players, work. This is closer to the mark.
Sending email and playing games are made possible by programs running on computers. The computer itself is like a newborn baby. It doesn’t really know how to do much. We hardly ever think about the basic machinery of human beings, because we mostly interact with the personalities that are running on that basic machinery, just like programs running on computers. For example, when you’re on a web page, you’re not reading it using just the computer itself; you’re reading it using programs someone else wrote that are running on your computer, the computer hosting the web page, and all of the computers in between that make the internet function.
What Is Computer Programming?
Teachers are people who train the basic human machinery to perform certain tasks. Similarly, programming is about becoming a teacher of computers. Programmers teach computers to do what the programmers want them to do.
Knowing how to teach computers is useful, especially when you want a computer to do something that it doesn’t know how to do and you can’t just go buy a program for it because nobody has created one yet. For example, you probably take the World Wide Web for granted, but it was invented not long ago, when Sir Tim Berners-Lee needed a better way for scientists at the European Organization for Nuclear Research (Conseil Européen pour la Recherche Nucléaire, or CERN) to share information. And he got knighted for it. How cool is that?
Teaching computers is complicated, but it’s easier than teaching people. We know a lot more about how computers work. And computers are a lot less likely to throw up on you.
Computer programming is a two-step process:
- Understand the universe.
- Explain it to a three-year-old.
What does this mean? Well, you can’t write computer programs to do things that you yourself don’t understand. For example, you can’t write a spellchecker if you don’t know the rules for spelling, and you can’t write a good action video game if you don’t know physics. So, the first step in becoming a good computer programmer is to learn as much as you can about everything else. Solutions to problems often come from unexpected places, so don’t ignore something just because it doesn’t seem immediately relevant.
The second step of the process requires explaining what you know to a machine that has a very rigid view of the world, like young children do. This rigidity in children is really obvious when they’re about three years old. Let’s say you’re trying to get out the door. You ask your child, “Where are your shoes?” The response: “There.” She did answer your question. The problem is, she doesn’t understand that you’re really asking her to put her shoes on so that you both can go somewhere. Flexibility and the ability to make inferences are skills that children learn as they grow up. But computers are like Peter Pan: they never grow up.
Computers are also like young children in that they don’t know how to generalize. They’re still useful because once you figure out how to explain something to them, they’re very fast and tireless about doing it, though they don’t have any common sense. A computer will tirelessly do what you ask without evaluating whether it’s the wrong task, much like the enchanted broomsticks in “The Sorcerer’s Apprentice” segment of the 1940 movie Fantasia. Asking a computer to do something is like asking the genie from a magic lantern (not the FBI version) to grant a wish. You have to be really careful how you phrase it!
You may doubt what I’m saying here because computers seem more capable than they are. When you use a computer, for example, it knows how to draw pictures, correct your spelling, understand what you’re saying, play music, and so on. But keep in mind, that’s not the computer—it’s a complicated set of computer programs that someone else wrote that allows the computer to do all of those tasks. Computers are separate from the programs that run on them.
It’s like watching a car on the road. It seems pretty good at stopping and starting at the right times, avoiding obstacles, getting where it’s going, eating when it gets hungry, and so on. But it’s not just the car. It’s the car and the driver packaged together. Computers are like the cars, and programs are like the drivers. Without knowledge, you can’t tell what’s done by the car and what’s done by the driver. (See “Southbound on the Freeway” by May Swenson. You might change your answer to the question posed at the end of the poem during your lifetime.)
In sum, computer programming involves learning what you need to know to solve a problem and then explaining it to a young child. Because there are lots of ways to solve a problem, programming is just as much an art as it is a science. It involves finding elegant solutions as opposed to using brute force. Yes, you can get out of a house by bashing a hole in the wall, but it’s probably a lot easier to go out the door. Many can write something like HealthCare.gov in millions of lines of code, but it takes skill to do it in thousands of lines.
Before you can instruct a three-year-old, though, you need to learn about three-year-olds and what they understand. And this isn’t any ordinary three-year-old—it’s an alien life form. A computer doesn’t play by the same rules that we do. You may have heard of artificial intelligence (AI), which tries to get computers to act more like people. Progress in that field has moved much slower than originally anticipated. That’s mainly because we don’t really understand the problem; we don’t know enough about how humans think. As you can imagine, it’s pretty hard to teach an alien to think as we do when we ourselves don’t know how exactly it’s done.
The human brain lets you do things without consciously thinking about them. Your brain started out as just a piece of hardware, which then got programmed. For example, you learned to move your fingers and then you learned to grab things. After practice, you can just grab things without thinking about all the steps that make it possible. Philosophers such as Jean Piaget (French psychologist, 1898–1980) and Noam Chomsky (American linguist born in 1928) have developed different theories about how this process of learning works. Is the brain just a general piece of equipment, or does it have special hardware for functions like language? This issue is still being studied.
Our incredible ability to perform tasks unconsciously makes learning how to program difficult, because programming requires breaking down tasks into smaller steps that a computer can follow. For example, you probably know how to play tic-tac-toe. Get a group of people together and have each of you independently list the steps a player should take in order to make a good move for any configuration of the board. (I’m sure you can find this online, but don’t look it up.) After everybody has made their lists, hold a playoff. Find out whose rules rule! How good were your rules? What sort of things did you miss? Do you actually know what you’re doing when you play the game? Chances are, there were a number of factors that you didn’t think to spell out because you understand them intuitively.
In case it’s not obvious, the first step, understanding the universe, is much more important than the second, explaining it to a three-year-old. Think about it: what good is it to know how to talk if you don’t know what to say? Despite this, current education focuses on the second step. This is because it’s much easier to teach and grade the mechanical aspects of the task than the creative elements. And in general, teachers have little training in the field and are working from curricula developed elsewhere that they were given to use. This book, however, focuses on the first step. While it can’t cover the universe in general, it examines problems and their solutions in the computer universe instead of dwelling on the exact programming syntax needed to implement those solutions.
Coding, Programming, Engineering, and Computer Science
A number of different terms are used to describe working with software. These terms have no exact definitions, although they have acquired some rough meanings.
Coding, a fairly recent term popularized as part of “learning to code,” can be viewed as the somewhat mechanical work of translation. Let’s compare it to the job of medical coding. When you visit a doctor, getting a diagnosis is the easy part. The hard part is translating that diagnosis into one of the over 100,000 codes in the ICD standards, ICD-10 at the time of writing. A Certified Professional Coder who has learned these codes knows that, when a doctor comes up with a diagnosis of “struck by cow,” it should be assigned code W55.2XA. This is actually harder than many coding jobs in the programming space due to the sheer number of codes. But the process is similar to what a coder would do if directed to “make that text bold” on a web page; a coder knows which code to use to make that happen.
The ICD-10 standard is so complicated that few coders know it all. Instead, medical coders get certified in specialty areas such as “Diseases of the nervous system” or “Mental and behavioral disorders.” This is analogous to a coder being proficient in a language such as HTML or JavaScript.
But programming—that is, being a programmer—means knowing more than a specialty area or two. The doctor in this scenario is analogous to a programmer. The doctor determines a diagnosis by evaluating the patient. This can be pretty complex. For example, if a patient has burns and is soaking wet, is it a “bizarre personal appearance” (R46.1) or a “burn due to water skis on fire, initial encounter” (V91.07XA)? Once the doctor has a diagnosis, a treatment plan can be devised. The treatment plan must be effective; the doctor probably doesn’t want to see the same patient suffering from a bad case of “parental overprotection” (Z62.1).
Just like the doctor, a programmer evaluates a problem and determines a solution. For example, maybe there’s a need for a website that allows people to rank ICD-10 codes in terms of silliness. A programmer would determine the best algorithms for storing and manipulating the data, the structure of the communication between the web client and server, the user interface, and so on. It’s not a simple “plug in the code” sort of thing.
Engineering is the next step up in complexity. In general, engineering is the art of taking knowledge and using it to accomplish something. You could consider the creation of the ICD standards to be engineering; it took the large field of medical diagnoses and reduced them to a set of codes that could be more easily tracked and analyzed than doctor’s notes. It’s a matter of opinion as to whether or not such a complex system represents good engineering. As an example of computer engineering, many years ago I worked on a project to build a low-cost medical monitor such as those that you see in hospitals. The charge I was given was to make a system that a doctor or nurse could figure out how to use in less than 5 minutes without any documentation. As you might imagine, this required much more than just knowledge of programming. And I beat the goal—my solution ended up taking about 30 seconds to learn to use.
Programming is often confused with computer science. While many computer scientists program, most programmers aren’t computer scientists. Computer science is the study of computing. Computer science discoveries are used by engineers and programmers.
Coding, programming, engineering, and computer science are independent but related disciplines that differ in the type and amount of knowledge required. Being a computer scientist, engineer, or coder doesn’t automatically make someone a good programmer. While this book gives you a taste of how engineers and computer scientists think, it’s not going to make you one; that typically requires a college education combined with some hard-earned relevant experience. Engineering and programming are similar to music or painting—they’re part skill and part art. The exposition of both aspects in this book should help you to improve your skills as a programmer.
The Landscape
Computer design and programming is a huge field of study, which I won’t be able to cover here. You can visualize it in layers, as shown in Figure 1.
Figure 1: Computer landscape
Keep in mind that Figure 1 is a simplification and that the lines dividing the various layers are not as clean in reality.
The majority of people are users of computer systems. You’re probably in that camp right now. There are specialized sorts of users called system administrators whose job is to keep computer systems working. They install software, manage user accounts, do backups, and so on. They typically have special powers not granted to normal users.
The people who write programs like web pages, phone apps, and music players are called application programmers. They write the software that users use to interact with the computers, using blocks that others have created. Application programming is being taught in most “learn to code” classes as if all programmers have to learn is how to import these other blocks and glue them together. Although you can get away with that a lot of the time, it’s much better to actually understand both those blocks and the glue.
Application programs don’t talk to the computer hardware directly; that’s where system programming comes into play. System programmers make the building blocks used by application programmers. System programmers need to know about hardware because their code interacts with it. One of the goals of this book is to teach you things you need to know in order to be a good system programmer.
Computer hardware includes not only the part that does the actual computing but also how that part connects to the world outside. Computer hardware is expressed as logic. It’s the same logic used to write computer programs, and it’s key to understanding the workings of the computer. The logic is constructed from various types of electronic circuits. Circuit design is beyond the scope of this book, but you can learn more about it by studying electrical engineering. Consider a double major in electrical engineering and computer science if you want to rule the world.
Of course, basic science underpins it all, providing everything from our understanding of electricity to the chemistry needed to create chips.
As shown in Figure 1, each level builds on the one beneath it. This means that poor design choices or errors at lower levels affect everything above. For example, a design error in Intel Pentium processors circa 1994 caused some division operations to produce incorrect results. This affected all software that used floating-point division in these processors.
As you can see, system programming is at the bottom of the software hierarchy. It’s similar to infrastructure, like roads, electricity, and water. Being a good programmer always matters, but it matters more if you’re a system programmer, because others rely on your infrastructure. You can also see that system programming is sandwiched between application programming and computer hardware, which means you need to learn something about both of those. The Sanskrit word yoga translates to “union,” and just as yoga practitioners seek to unify their mind and body, system programmers are techno-yogis who unify the hardware and software.
You don’t have to learn system programming in order to work at one of the other levels. But if you don’t, you’ll have to find someone else to help you deal with issues out of your domain rather than being able to figure them out for yourself. An understanding of the core technology also leads to better solutions at higher levels. This isn’t just my opinion; check out the 2014 blog post “The Resource Leak Bug of Our Civilization” by Ville-Matias Heikkilä for a similar view.
This book also aims to cover a lot of retro history. Most programmers aren’t learning the history of their craft because there is so much material to cover. The result is that a lot of people are making mistakes that have already been made. Knowing some of the history allows you to at least make new and better mistakes rather than repeat past ones. Bear in mind that the hot new technology you’re using today will quickly become retro tomorrow.
Speaking of history, this book is jam-packed with interesting technologies and the names of their inventors. Take some time to learn more about both the technologies and the people. Most of the people mentioned solved at least one interesting problem, and it’s worth learning about how they perceived their world and the way in which they approached and solved problems. There’s a great exchange in Neal Stephenson’s 2008 novel Anathem:
“Our opponent is an alien starship packed with atomic bombs. We have a protractor.”
“Okay, I’ll go home and see if I can scrounge up a ruler and a piece of string.”
Note the reliance on fundamentals. It’s not “Let’s look up what to do on Wikipedia” or “I’ll post a question on Stack Overflow” or “I’ll find some package on GitHub.” Learning to solve problems that nobody else has solved is a crucial skill.
Many of the examples in this book are based on old technology such as 16-bit computers. That’s because you can learn almost everything you need to know from them and they’re easier to fit on a page.
What’s in This Book
The book is conceptually divided into three parts. The first part explores computer hardware, both what it is and how it’s built. The second part examines the behavior of software running on hardware. The last part covers the art of programming—working with others to produce good programs.
Chapter 1: The Internal Language of Computers
This chapter starts exploring the three-year-old mentality. Computers are bit players; they herd bits for a living. You’ll learn what they are and what can be done with them. We’ll play make-believe to ascribe meaning to bits and to collections of bits.
Chapter 2: Combinatorial Logic
This chapter examines the rationale for using bits instead of digits and explores the justification for digital computers. This includes a discussion of some of the older technologies that paved the way for what we have today. It covers the basics of combinatorial logic. You’ll learn how to build more complicated functionality from bits and logic.
Here you’ll learn how to use logic to build memory. This includes learning how to generate time, because memory is nothing but state that persists over time. This chapter covers the basics of sequential logic and discusses various memory technologies.
This chapter shows how computers are constructed from the logic and memory elements discussed in the earlier chapters. A number of different implementation methodologies are examined.
Chapter 5: Computer Architecture
In this chapter, we’ll explore some of the add-ons to the basic computer that we saw in Chapter 4. You’ll learn how they provide essential functionality and efficiency.
Chapter 6: Communications Breakdown
Computers need to interact with the outside world. This chapter covers input and output. It also revisits the difference between digital and analog and how we get digital computers to work in an analog world.
Now that you’ve seen how computers work, we’ll look at how to use them effectively. Computer programs manipulate data in memory, and it’s important to map the way memory is used to the problem being solved.
Chapter 8: Language Processing
Languages have been invented to make programming computers easier for people. This chapter looks at the process of converting languages into something that actually runs on computers.
A lot of programming is done for web browsers. This chapter looks at how a web browser works and teases out its main components.
Chapter 10: Application and System Programming
In this chapter, we’ll write two versions of a program that runs at two of the different levels from Figure 1. The chapter exposes many of the differences between application- and system-level programming.
Chapter 11: Shortcuts and Approximations
Making programs efficient is important. This chapter explores some of the ways in which we can make programs more efficient by having them avoid unnecessary work.
Chapter 12: Deadlocks and Race Conditions
Many systems include more than one computer. This chapter examines some of the problems that can occur when we’re trying to get computers to cooperate.
Computer security is an advanced topic. This chapter covers the basics while punting on the heavy math.
Chapter 14: Machine Intelligence
This chapter also covers an advanced topic. New applications result from the combination of big data, artificial intelligence, and machine learning—from driving your car to driving you nuts with advertisements.
Chapter 15: Real-World Considerations
Programming is a very methodical and logical process. But humans are involved in the determination of what and how to program, and humans are often lacking in logic. This chapter discusses some of the issues of programming in the real world.
When reading this book, bear in mind that many of the explanations are simplified and therefore not correct down to the smallest detail. Making the explanations perfect would require too much distracting detail. Don’t be surprised if you discover this as you learn more. You can consider this book to be a glossy travel brochure for a trip to computer-land. It can’t cover everything in detail, and when you go visit, you’ll find plenty of subtle differences.