When I originally outlined this chapter, I wanted to provide tips and best practices on building systems, but most advices that came to mind tended to be too domain specific or hyper specific to certain types of patterns that can appear in code. As I thought about it, I came up with four pieces of advice which can be applied to executing and delivering any project:

One aspect of software to think about during the design and building phases is the separation of concerns. The idea is that all of the code for dealing with certain types of data should be located together and provide an API, then any other software that wants to work with that data does so through the API. This means that your data is not touched by other code or services. In some cases, this is easy, but more often than not, making data more separated by an API can cause data duplication or poor performance. This is because distributed state management is difficult.

If four pieces of software need to know how much money a user has in their bank account, for example, querying the database may seem easier, but in the long term, calling a well-defined API will make your life easier. An example is changing the schema of how your data is stored. If you keep the API consistent, you can change how the data is stored underneath, without having to update client applications. If you do want to make performance improvements, you can achieve significant gains by just updating the API interface, instead of having to improve every client that talks to your database.

If you do this separation well, then in theory, when needed, you can rip out a part and replace it with a service. A common example of this is authentication. Often teams start with authentication inside their application, but as other services start to need to do authentication, they replace the code in the app with a call to an external service, which presents a similar API.

A somewhat similar idea is the Unix philosophy. This philosophy has a few tenets, but the two that I like to focus on are writing software that does one thing and writing software that works together. You can see this in most common Linux command-line programs. For example, you can take the output of cat and sort it with the sort command. cat file{1,2,3}.txt | sort > sorted.txt takes three files (file1.txt, file2.txt and file3.txt), merges their output into a single stream and then sends that stream to sort, which sorts all of the incoming data, then is written to sorted.txt. The point is that cat, sort, and many other programs use standard input as their text input and standard output as their text output. Normally, a shell writes standard out to the screen, but by adding a pipe (|), your shell points the standard output of the first program into the standard input of the second program.

Note that separation of concerns and the Unix philosophy are often used as reasoning for moving to a service-oriented architecture or microservices. Both of these terms are the idea of separating your code into lots of smaller services or pieces. There are problems with such an infrastructure (more pieces to monitor, possibly more expensive and a higher possibility of a network failure), but usually separation of concerns is not a great argument if it is the sole argument to move. You can separate concerns inside your code if you want, just using separate classes or other organizational techniques.

Another factor that we should probably talk about in regard to development is long-term planning and thinking. So much of modern development is focused around fast iteration, that often people do not remember to look at where they have gone and where they are going. I like to do a quarterly retrospective and plan for myself to solve this. Some businesses have this built into their planning process, but you can also do it for yourself.

A common framework that some businesses use, I also use for my personal planning. It is called objectives and key results (OKRs). The idea behind OKRs is that you take a list of things that you want to achieve, which are objectives, and then create measurable checkpoints as your key results. Then, at the end of the quarter, you look at your OKRs and look at what you planned on doing, before comparing them to what you completed. In a larger, more formal organization, you may score your progress based on the OKRs you completed, but for yourself, it is just a benchmark. Look at what you thought you were going to do and ask: what prevented you from completing some projects? What other things did you do that you didn't plan for? Did you want to do those things? If not, can you make goals for the next quarter to remove them by building a tool to make them easier, automating them, removing the need for them, and so on. Then, looking forward, create new or adjusted OKRs based on the next quarter. Often, I will create year-long goals. Some are work related, such as unifying how all of our services do job scheduling, while some are life related, such as losing 40 pounds of weight. In my current role, my OKRs are completely disconnected from my management structure. So even though my boss is horrible at doing performance reviews, I can still do self-reflection and track progress for both my work and personal life.

Sometimes looking for future ideas can be difficult. I keep a notebook where I write random things in throughout the day. Sometimes I put down ideas that I want to build in the future. Some are one sentence, while others are bullet lists of features and some are very rough infrastructure and architecture sketches or wireframes of user interfaces.

Figure 2: In this photograph, I have sketched out the rough design for a blog. The frontend, or FE, is built in JavaScript using React and Next.js for rendering content. Apollo is used to query the backend (or BE). The backend is a Go app with one path, /graphql, which controls the data. It sits on top of a PostgreSQL database for storing state. There is also a cron job that gets data from an external service, Pinboard, and makes queries to the GraphQL endpoint.

Notebooks are also great places to record how you are feeling, things you want to do, and things you want to get off your chest. A classic tool is to write a letter to a person you are angry with and then never do anything with it. I used to write these in emails, often in an inebriated state and still ended up sending a few, which often got me into trouble or hurt a relationship with someone when I didn't need to do that. By writing it on paper, that never happens! There's no way that I can accidentally send an email when it's on paper.