A long time ago (in a galaxy not too far away), regular people started connecting computers together. A few brave souls tried to do this with dial-up 1200-baud modems over phone lines. Pioneers brought up Bulletin Board Systems (BBSs ; message boards that one person at a time could dial into and exchange short messages, and later small files, with each other). I brought up the eighth BBS in the world, in Atlanta, in about 1977, using code from the original CBBS in Chicago (created by Ward Christensen and Randy Suess). I used a modem donated by my friend Dennis Hayes (of Hayes Microcomputer Products). Later there were thousands of online Bulletin Board Systems, all over the world. Soon there followed commercial “information utilities” like CompuServe 1 and The Source,2 which were like giant Bulletin Board Systems (BBSs) with many more features. Tens of thousands of users could connect to these simultaneously. It was like the first crude approximation to the Internet of today, based on circuit-switched connections over telephone lines. Everything was text oriented (non-graphical) and very slow. 1200 bits/second was typical at first, although later modems with speeds of 2400 bits/second, 9600 bits/second, 14.4 Kbps, 28.8 Kbps, and finally 56 Kbps were developed and came into widespread use. Later these modems were primarily used to dial into an ISP to connect to the Internet, and some people are still using them this way.
Real Computer Networking
While home computer users were playing around with modems and Bulletin Board Systems, the big computer companies were working on ways to connect “real” computers at higher speeds and with much more complex software.
Ethernet and Token Ring
Much of this was based on Ethernet ,3 which was created by a team at Xerox Palo Alto Research Center (PARC 4) led by Robert Metcalfe 5 between 1973 and 1975. The first released version (1976) ran at 3 Mbps. Metcalfe left PARC in 1979 to create 3Com 6 and create commercial products based on Ethernet. Working together with Digital Equipment Corporation (DEC), Intel, and Xerox (hence the “DIX”7 standard), 3Com released the first commercial products running at 10 Mbps. Ethernet was standardized in 1980 by the IEEE (Institute of Electrical and Electronics Engineers) as 802.3. Early versions ran on 10base2 (a small-diameter coax cable) or 10base5 (a larger-diameter coax cable). These used a “multidrop” architecture, which was subject to many reliability problems. With the introduction of the simpler to deploy and manage “unshielded twisted pair” (UTP) (actually four-pair, or eight-wire) cables (known as 10baseT, mid-1980s) and star architectures using “hubs” and later “switches,” Local Area Networks (LANs) really took off. Today, virtually all Ethernet networks use twisted pair copper wire (up to gigabit speed) or fiber-optic cable (for higher speed and longer runs). I helped deploy a 10base2 coax Ethernet network in Hong Kong in 1993. Trust me, twisted pair cabling is a lot easier to work with.
IBM for many years pushed a competing physical layer network standard called “Token Ring”8 (later standardized as IEEE 802.5). Token Ring was available in 4 Mbps and 16 Mbps versions. Later, a 100 Mbps version was created, but by then Ethernet dominated the market, and Token Ring quietly died. FDDI 9 (Fiber Distributed Data Interface) still in use today is based on the Token Ring concept.
Network Software
Network software quickly evolved once Ethernet and Token Ring hardware became available. One of the main goals was to “hide” the differences between various hardware-level technologies (Ethernet, Token Ring, Wi-Fi, etc.) and the higher-level software. This led to the multiple layers of the network stack. The bottom layer is very hardware specific, and the upper layers introduce more and more hardware independence, so that applications can be written once and run over any hardware media.
DEC 10 was one of the first companies to create networking software with DECnet 11 (1975). IBM had System Network Architecture (SNA,12 1974). Xerox created the PARC Universal Packet protocol 13 (PUP protocol, late 1970s), which eventually evolved into Xerox Network Services 14 (XNS, early 1980s) at PARC. XNS was the basis for the late Banyan VINES15 network OS, based on “VINES IP” (similar to but incompatible with IPv4 from TCP/IP). Banyan VINES included the first network directory service, called “StreetTalk .”16 XNS also was the basis for Novell NetWare 17 (IPX/SPX, 1983), which eventually added its own NetWare Directory Services (NDS,18 1993).
Microsoft worked with 3Com to create their own network OS, called LAN Manager. It used the SMB 19 (Server Message Block ) protocol on top of either the NBF (NetBIOS 20 Frames) protocol or modified XNS. In 1990, Microsoft added support for TCP/IP as an alternate protocol (LAN Manager 2.0). With the release of Windows NT Advanced Server in 1993, Microsoft finally phased out LAN Manager. By Windows NT v3.51 21 (May 1995), Microsoft encouraged users to deploy only TCP/IP (4 years ahead of Novell’s support for TCP/IP). This lead time allowed Microsoft to take over leadership in personal computer networks from Novell. Microsoft introduced their version of network directory services in Windows Server 2000, now known as Active Directory.22 The SMB protocol still survives as Microsoft’s “File and Printer Sharing” protocol (now layered on TCP/IP, instead of NetBIOS or XNS). An open source implementation of this is available as SAMBA.23
The Beginnings of the Internet (ARPANET)
While all this commercial activity was going on, the US military (at their Defense Advanced Research Projects Agency, or DARPA 24), with the help of Bolt, Beranek, and Newman (BBN 25) and Mitre,26 were designing a new, decentralized communication system based on packet switching. Existing communication systems (telephone, radio, etc.) were centralized and hence subject to being completely disabled due to the failure or loss of a few central nodes. Packet-switched networks were highly decentralized and had a fascinating new property, which is that you could lose large parts of a network and the remaining parts would still work (assuming at least some links connected the working parts).
The first network protocol developed as part of ARPANET was called the 1822 27 protocol (named after BBN Report 1822) and was implemented by a Network Control Program ,28 so the protocol was often referred to as NCP. The first email was sent over NCP in 1971, and the File Transfer Protocol followed in 1973. On January 1, 1983 (“flag day”), NCP was turned off officially, leaving only IPv4 on the Internet. I consider the NCP era to be the First Internet and the IPv4 era as the Second Internet. That makes the evolving IPv6 era the Third Internet. Fortunately, there is no need for a flag day to go from IPv4 to IPv6, as they can coexist (and probably will for perhaps 5–10 years). We learned not to do that from the NCP to IPv4 transition. They broke global email for months in that process.
In May 1974, Vint Cerf 29 and Bob Kahn 30 released the paper “A Protocol for Packet Network Interconnection.”31 This described a monolithic protocol called TCP that combined the features of both modern TCP and IPv4. Later Jon Postel 32 was instrumental in splitting apart TCP and IP as we know them today. Vint Cerf is today considered the “father of TCP/IP” and is now an “Evangelist” at Google. He understands very well the problems with the current implementation of IPv4 (and why these things were done). He advocates for users to migrate to IPv6, which restores his original concept of a flat address space (no NAT), where any node can connect directly to any other node. One of my proudest possessions is a copy of RFC 791 (IPv4) autographed in person by Vint Cerf.
If you’d like to read about the creation of the Second Internet, I recommend the book Where Wizards Stay Up Late: The Origins of the Internet,33 by Katie Hafner and Matthew Lyon. It is of considerable interest to those of us creating the Third Internet, as we are facing some of the same problems they did. Only this time around, we’ve got over a billion legacy users (and staggering investments in hardware and software) to worry about. On the other hand, we’ve got three decades of operational experience with IPv4 to draw upon.
Higher-level software protocols were built on top of the TCP and IP layers, called “application protocols ,” such as SMTP 34 and IMAP 35 (for email), FTP 36 (for file transfer), Telnet 37 (for terminal emulation), and more recently HTTP 38 (used in the World Wide Web) and SIP 39 and RTP 40 (used in VoIP). The resulting suite of protocols became known for its two most important protocols, TCP and IP, or TCP/IP 41 (its formal name is “the Internet Protocol Suite”).
UNIX
About this time (1973), Bell Labs (a research group within AT&T) created an interesting new operating system (called PWB-UNIX 42) and a new language (in which UNIX was written) called “C.”43 Because of a 1958 consent decree, AT&T as a regulated monopoly was not allowed to market or sell UNIX commercially. They licensed it (complete with source code) to several universities. One of these was the University of California at Berkeley 44 (UCB; also famous for being the center of campus-based communist student activities at the time). The team at UCB extended UNIX in several very important ways such as adding Virtual Memory. They also integrated the new network protocol from ARPA as the first commercial implementation of TCP/IP. The “Berkeley System Distribution”45 of UNIX became a main branch. Over time, they rewrote most of it and wanted to release it for free. AT&T sued them in court, and it seems most of the examples of “stolen code” AT&T cited had actually been written at UCB. The judge ruled that if UCB rewrote the remaining 10% or so (so there was zero original AT&T code), they could release that. That rewrite became 386BSD,46 the starting point for FreeBSD 47 (the first open source operating system). Later FreeBSD was chosen by Japan’s Kame48 project to deploy the first version (the “reference” implementation) of a IPv6 network stack, in an eerie echo of BSD UNIX’s choice for the first commercial IPv4 implementation.
UNIX and TCP/IP became very popular on college campuses, and with high-end workstation vendors, such as Sun, Silicon Graphics, and Intergraph. Personal computers were not powerful enough to run UNIX until the Intel 386, at which point UCB ported the BSD version to the 386. However, as documented above, most personal computer networking was already moving to TCP/IP.
Open System Interconnection (OSI)
Because “International Organization for Standardization” would have different acronyms in different languages (IOS in English, OIN in French for Organisation internationale de normalisation), our founders decided to give it the short form ISO. ISO is derived from the Greek isos, meaning equal.
This is where the famous “seven-layer” network model comes from (TCP/IP is really based on a “four-layer” model, which has caused no end of confusion among young network engineers). At one point the US government decided to officially adopt OSI for its networking (this was called GOSIP,52 or Government Open Systems Interconnection Profile , defined in FIPS 146-1, 1990). Unfortunately, OSI was really more of an academic specification, not a real working network system, like TCP/IP was. After many years, GOSIP was finally abandoned, and IPv4 was deployed, but GOSIP’s legacy has hindered the adoption of IPv6 in the United States (“Here we go again – GOSIP phase 2!”). X.400 53 email and X.500 54 directory systems were built on top of OSI and will not run on TCP/IP without substantial compatibility layers. One small part of X.500 (called X.509,55 “The Authentication Framework”) was the source of digital certificates 56 and Public Key Infrastructure,57 still used today. Lightweight Directory Access Protocol (LDAP 58) was an attempt to create an X.500-like directory system for TCP/IP-based networks. That’s about all that is left of the mighty OSI effort today, outside of computer science textbooks and Cisco Press books.
IPv6 was based heavily on IPv4 and was defined by the same group that defined IPv4 (the IETF 59). It is the “natural” and straightforward evolutionary step after IPv4. At this point everyone has decided that IPv6 is inevitable, although there were many battles and brave resistance for years.
Email Standardization
By this time, essentially all computer vendors had standardized on TCP/IP, but there were still a lot of competing standards for email, including Microsoft’s MS-Mail, Lotus’s cc:Mail, and MCI Mail. The Internet folks used a much simpler email standard called SMTP 60 (Simple Mail Transfer Protocol). It first became the connecting backbone between various email products (everyone had their email to SMTP gateways, so users could exchange messages across organizations). Soon, everyone started using SMTP (together with POP3 and later IMAP) all the way to the end user. Today virtually all email worldwide is based on SMTP and TCP/IP. However, Microsoft is forcing their own proprietary protocols (EWS and now Graph) into this space with Office 365. By the way, most of Office 365 works fine over IPv6. Teams and Azure are still IPv4-only (at least VMs in Azure are IPv4-only).
Evolution of the World Wide Web
Several other Internet applications evolved, including WAIS (Wide Area Information Server , for storing and retrieving documents) and Archie 61 (the very first search engine). In turn, these efforts were merged with the idea of Hypertext (documents with multilevel links) and evolved into HTML (Hypertext Markup Language) and HTTP (Hypertext Transfer Protocol). The World Wide Web was off and running. The first web browser and web server were created at the National Center for Supercomputing Applications 62 (NCSA) at the University of Illinois, Urbana-Champaign campus. The people who created those software projects (primarily Marc Andreessen 63 and Eric Bina 64) were soon hired by Jim Clark,65 one of the founders of Silicon Graphics, to start Netscape,66 one of the most successful and important companies in the Second Internet. They created a new and more powerful web server (Netscape Application Server 67) and web browser (Netscape Navigator 68). As an interesting aside, the original browser created by Andreessen at NCSA (Mosaic 69) later became the starting point for Microsoft’s Internet Explorer web browser.
And That Brings Us Up to Today
That pretty much brings us up to the present day where the entire world has standardized on IPv4 for both LANs (Local Area Networks) and WANs (Wide Area Networks). Multiprotocol Label Switching (MPLS) is not a competitor to TCP/IP; it is one more alternative at the Link Layer, peer to Ethernet and Wi-Fi. More and more companies and organizations built TCP/IP networks and connected them together to create the Internet. Major telcos provided “backbone” WAN connections and dial-up access service (soon known as ISPs or Internet Service Providers 70). As the number of users (and the amount of traffic) on the Internet grew exponentially, Internet Exchange Points (IXPs) 71 were created around the world. These are places where ISPs connect to each other so that traffic from a user of any provider can reach users of any other provider, worldwide.
If you’d like to understand more about the physical implementation of the Internet, see Tubes: A Journey to the Center of the Internet ,72 by Andrew Blum. Highly recommended.
Summary and a Look Ahead
In this chapter, we covered the history of computer networking from the early days of modems and BBSs then to “real” early computer networking (DECnet, XNS, SNA, NetBIOS, NetWare, etc.) up to the consolidation of multiple network protocols into a single standard, which was TCP/IP (with version 4 of IP, or IPv4). We also covered the “precursor” to the IPv4-based Internet, which was called ARPANET, based on the host-host protocol (or NCP).
We covered some of the key participants (Bob Kahn, Vint Cerf) who created the technology used in the Second Internet and companies who pioneered the most widely used protocols, hardware, and applications (e.g., 3Com, Netscape).
Today, the most widely used application on the Second Internet is the World Wide Web. This evolved due to the widespread deployment of NAT and private addresses. These limited most users to only being able to make outgoing connections to a small number of centralized serves (that have public IP addresses), thereby centralizing functionality. This is comparable to dumb terminals (browsers) being used to access central mainframes (web servers) where the actual computing takes place.
In the Third Internet, there is no shortage of public IP addresses or need for NAT – even phones can now have public IP addresses and host servers or do end-to-end direct connections. This will lead to extensive decentralization. Now people can take advantage of the amazing computing power in their desktops, laptops, and even phones, rather than centralized computing on web servers. Rather than sending the entire GUI over the network (HTTP/HTML), we will run native GUI applications on our devices and exchange only data over the network. There will still be a need for shared databases, but web browsers and servers will decrease in importance. Zero-Trust Networks are a start in this direction. This is a major paradigm shift made possible by IPv6.
The original concept of the Internet was “complexity at the edge, simplicity at the core” – the World Wide Web has stood that on its head, primarily because of the limitation of IPv4 with NAT and private addresses.