Chapter 3
Write a Log Package

In this book we’re building a distributed service to learn how to create distributed services with Go (shocker). But how does building a log in this chapter help us achieve that goal? I believe the log is the most important tool in your toolkit when building distributed services. Logs—which are sometimes also called write-ahead logs, transaction logs, or commit logs—are at the heart of storage engines, message queues, version control, and replication and consensus algorithms. As you build distributed services, you’ll face problems that you can solve with logs. By building a log yourself, you’ll learn how to:

• Solve problems using logs and discover how they can make hard problems easier.

• Change existing log-based systems to fit your needs and build your own log-based systems.

• Write and read data efficiently when building storage engines.

• Protect against data loss caused by system failures.

• Encode data to persist it to a disk or to build your own wire protocols and send the data between applications.

And who knows—maybe you’ll be the one who builds the next big distributed log service.