Chapter 11. Onward and upward

This chapter covers

• Choosing which avenues to explore next in your software development career
• Developing a plan of action for continued learning

Believe it or not, you’ve reached the last chapter of this book. Wasn’t that fun? You’ve learned many of the facets of thoughtful software design in this book, but there’s a whole world out there still to discover. It can be difficult to figure out what comes next. If you’re feeling unsure about which trajectories to explore, read this chapter for some strategies and topic ideas.

11.1. What now?

As you gain experience, you’ll continue learning a great deal. You’ll also encounter things you want to learn but don’t have the time or experience to cover just yet. There will also be an ever-present and near-infinite set of things you aren’t aware of at all. These are concepts that either haven't occurred to you yet or you don't have the words to express.

Donald Rumsfeld succinctly (and humorously) put it like this:

There are known knowns—there are things we know we know. We also know there are known unknowns—that is to say, we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know.

Donald Rumsfeld

Being an effective engineer rarely amounts to being exhaustively knowledgeable on a subject. More often, you can work effectively by knowing the right thing to look up and which resources are available to you. In short, resourcefulness is more valuable than experience.

As you grow, you’ll probably amass a list of blog posts, tools, and topics you’re interested in. You’ll also learn new things out of necessity as you build software. Eventually, when you decide it’s time to dig into some of these new topics, it can help to set yourself up for success with a learning plan.

11.1.1. Develop a plan

Have you ever gone down the Wikipedia rabbit hole? You start reading about a topic, and suddenly it’s 2:37 A.M., and you have 37 tabs open in your browser. You click through to links of interest, sometimes going several layers deep down a particular path. Although you might feel like you’ve wasted your evening, this turns out to be an effective strategy for uncovering information.

A mind map organizes information in a hierarchical structure you can explore visually. A mind map starts from a central node—an overall concept you’re interested in learning. It then branches out, with each node representing subtopics or related concepts to explore—like when you couldn’t help but click the link to “Cosmic latte” from the page about “Beige.” By using a mind map to enumerate things you want to learn about, you can build up a pretty good picture of different areas you’ll need to cover.

If you want to learn about natural language processing, you might draw a mind map like the one shown in figure 11.1. A few high-level categories of activities eventually branch into specific and involved topics like lemmatization and Markov chains. Some of these may be things you’ve heard of but know little about, but you should still write them down. Even if you don’t know what branch a topic falls under, you’ll eventually figure out a path to it as you learn more about the topics around it.

Figure 11.1. A mind map for learning about natural language processing

This visual representation helps emphasize relationships between topics, which can help you retain the information you learn. It also acts as a kind of map in the traditional sense; concepts become regions of the map, and you can see which areas are charted well or remain unexplored. This comes in handy as you work on learning more.

If you don’t have enough experience with a topic to draw a full map, don’t worry. Writing down a short list of things can still be effective. The key is to have something you can refer to that reminds you what you’ve already done and what’s left.

Once you’ve got your next steps mapped out, you’re ready to do some learning.

11.1.2. Execute the plan

With your learning topics mapped out (or listed), you can start exploring the resources available to you. These may be books, online courses, or a friend or colleague with experience in the topic. Figure out your learning style too. Some people can learn just by reading, whereas others need to write some real code and see some real output for things to click. Be creative.

A mind map can work well because you can explore it nonlinearly. If you’re still familiarizing yourself with the terminology and concepts, you might first explore the things one level out from the center, as shown in figure 11.2. This can help you get the lay of the land, which will help you build some footing as you choose what to learn next. After you’ve got your bearings, you can choose a topic you find particularly riveting to explore more deeply, as shown in figure 11.3. The influx of new information about something that excites you is invigorating.

Figure 11.2. Exploring the breadth of a topic first

Figure 11.3. Exploring a single topic in depth

Tip

A common pitfall is to do a deep dive into one topic without enough context about the rest of the bigger picture, so make sure you maintain a balance. Too much focus in one spot too soon can lead you to solidify an inaccurate or incomplete understanding that can inhibit future learning.

Successful learning requires an iterative approach—as you gain more experience with a topic, you’ll naturally find more things to add to your mind map (or list). Adding things as you go along is perfectly fine, but make sure you feel comfortable with the topics you’re already learning before expanding into new ones. It’s easy to spread yourself too thin!

Keep it all sorted by tracking your progress.

11.1.3. Track your progress

Learning is subjective, so don’t expect that you’ll be able to say you’re “done” with most things. There are a few distinct states in learning about a particular topic:

1. Want or need to learn—It’s on your list of topics to cover, but you haven’t started on it yet.
2. Actively learning—You’ve explored and read some resources on the topic, and you’re looking for more.
3. Familiar—You understand the topic generally, and you have some idea how you might apply it.
4. Comfortable—You’ve applied the concepts from this topic a few times and have a handle on it.
5. Proficient—You’ve applied the concepts enough to know some of the nuances, and you know which resources to reach for when you encounter new kinds of problems.

Many expertise categorizations break these states down further, but each of these levels represents an observable shift in your behavior. It’s good to recognize which level you’re at so you can better understand which topics you want to invest your time in. You might not even want to reach “proficient” for topics you encounter only rarely or that don’t align with the tasks you want to accomplish. Explicitly writing this down will help you keep your plan up to date, as shown in figure 11.4.

Figure 11.4. Keeping track of your learning progress for each topic

You’ll probably learn several pertinent points about a topic at each level of your learning. They may not be large enough to justify adding more nodes to your mind map (although mind-mapping software makes this a low-cost activity), but it helps to write them down. You can use these notes to gauge what level of learning you’re at on a topic, and they may prompt you to revisit ideas that need more work.

I struggled for a long time in school and my career, retaining information only after significant repetition. Mapping things out and tracking my progress proved to be an effective aid in learning many of the ideas expressed in this book and beyond. If you haven’t tracked your learning like this before, give it a try.

With a framework for exploring and learning new ideas fresh in your mind, read on for some suggestions about where to go after you’re finished with this book.

11.2. Design patterns

Over the last several decades, developers have solved the same problems many times over. Looking across all these solutions, certain patterns have emerged. Some of these patterns provide loose coupling and extensibility, but others don’t.

These software design patterns are tried-and-true solutions, and naming them allows us to talk about them more concretely. A ubiquitous language, or shared vocabulary for the concepts a team needs to understand, goes a long way toward achieving the outcomes a team seeks.

You used a design pattern when you created the commands for Bark. The command pattern, as it’s known, is used frequently in applications like Bark to decouple the code that requests an action from the action itself. The command pattern always has a few common pieces, regardless of the situation in which it’s used:

1. Receiver—The entity that takes an action, like persisting data in a database or making an API call
2. Command—The entity that contains the info needed for the receiver to take its action
3. Invoker—The entity that triggers the command to alert the receiver
4. Client—The entity that assembles the invokers, commands, and receivers to achieve a task

In Bark, these pieces are as follows:

1. The PersistenceLayer classes are the receivers. They receive enough information to store or retrieve data (from a database, in the case of the BookmarkDatabase).
2. The Command classes are the commands. They store the information needed to communicate with the persistence layer.
3. The Option instances are the invokers. They trigger a command to take place when a user chooses an option in the menu.
4. The client module is the client. It hooks up options to commands properly so that users’ menu choices ultimately result in the desired action.

A unified modeling language (UML) diagram of these classes is shown in figure 11.5.^[1] UML diagrams are a common way of depicting the relationships between entities in a program. This book has been intentionally light on UML because it can add to the learning curve for the untrained eye. As you learn about design patterns, though, you’ll see UML diagrams come up frequently. Remember that the patterns themselves are what’s important to understand—if UML diagrams don’t work well for you, read about them instead.

¹

For more on UML, see Wikipedia’s “Unified Modeling Language” article: https://en.wikipedia.org/wiki/Unified_Modeling_Language.

Figure 11.5. The command pattern as used in the Bark application

11.2.1. Ups and downs of design patterns in Python

You’ve seen some of the benefits of using a specific design pattern in Python. The command pattern helped you decouple layers of abstraction in Bark, leading to flexible persistence, business logic, and presentation. Many of the other patterns you will learn may provide value as well.

To understand which design patterns you should learn and apply in Python, it’s important to understand the context in which many design patterns were developed and used. One significant driver for some design patterns is the language or languages from which they emerged. A number of design patterns come from Java, a statically typed language. Because of static typing, languages like Java are intentionally limited in how they can create instances of classes and so on. As a result, a number of design patterns are creational ones. Python’s dynamic typing frees it from many of these limitations, so many creational patterns simply aren’t necessary in Python.

Ultimately, as with many topics in this book, design patterns are a tool to help you get your work done. If you’re trying to use a design pattern to approach a problem, and it feels forced, it’s okay to move ahead without a specific pattern. A better pattern might jump out at you in the meantime.

The canonical reference for learning more about design patterns is Design Patterns: Elements of Reusable Object-Oriented Software.^[2] The online software development community also has many discussions on the topic, often with useful case studies that can help you further understand if and when to use a particular pattern.

²

Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley Professional, 1995).

11.2.2. Terms to start with

You can start your research into design patterns with the following terms:

• Design patterns
  • Creational design patterns
  • Factories
  • Behavioral design patterns
  • Command pattern
  • Structural design patterns
  • Adapter pattern

11.3. Distributed systems

In modern web-application development, you may need a server that handles HTTP traffic, a database to persist data, a cache to store frequently accessed data, and so on. These elements form a system—a group of interconnected pieces that make up a whole. The pieces of this system are frequently located on distinct machines, in separate data centers, and sometimes even on different continents, as shown in figure 11.6. These distributed systems add layers of value, complexity, and risk for developers to understand and address.

Figure 11.6. A system distributed across several locations

Some of the more interesting complexities that distributed systems exhibit are the ways in which they fail.

11.3.1. Modes of failure in distributed systems

Even on a single machine, a program may crash unexpectedly. Other programs that expect that program to be running may also crash if they haven’t accounted for the situation.

Chipping off pieces of an application to put them in new locations introduces new, exotic modes of failure. All applications may be running properly, but the network connection between them may fail. Most applications may be able to access a database, but one may not. Distributed systems techniques seek to withstand and recover from these modes of failure.

I’ve found that thinking about the ways a distributed system can fail is similar to thinking about functional testing. In chapter 5, you learned about creative exploratory testing as a way to enumerate as many facets of vulnerability as possible. Distributed systems require this same mindset on a larger scale because there are more moving parts.

11.3.2. Addressing application state

A big question in distributed systems is how to handle a part of the system crashing. You may be able to live without some pieces of the system, carrying on without the data they provide. Other pieces of the system may be necessary but not time-sensitive, so requests to them while they’re down can be stored and deferred until they’re back up. Remaining pieces of the system are critical to operations—the system comes to a halt without them. These are single points of failure.

Distributed systems are designed to minimize the single points of failure, favoring graceful degradation—carrying on without a particular action or information. Tools like Kubernetes (https://kubernetes.io/) augment the approach to processing failure through eventual consistency, which enables you to define the state you want for your system, providing a guarantee that the system will eventually reach the defined state. Pairing graceful degradation with eventual consistency leads to malleable systems that go down less often.

Although distributed systems aren’t new, there have been many recent developments in tooling and philosophy. Kubernetes and the ecosystem around it can certainly be applied to small systems as you learn, but it shines on larger, complex systems. You may want to start with the principles and techniques and then get some practice building a few distributed systems before moving into specialized tools.

11.3.3. Terms to start with

You can begin your research into distributed systems with the following terms:

• Distributed systems
  • Fault tolerance
  • Eventual consistency
  • Desired state
  • Concurrency
  • Message queueing

11.4. Take a Python deep dive

This may seem obvious, but another area you can keep growing in is Python. Although this book used Python in its examples to convey ideas about software design, there’s much to learn about the features, syntax, and power of the Python language.

11.4.1. Python code style

As you work more in Python, you’ll eventually get a sense for the code formatting you like. You’ll write your code in that style because it will be easier for you to read later on. But when someone else who’s been following their own style reads your code, they might have a hard time understanding it. PEP 8, the Python Enhancement Proposal for a Python style guide, suggests a standard style for Python code formatting so that you don’t have to spend time agonizing over it.^[3] Tools like Black (https://github.com/psf/black) take these suggestions a step further, imposing a deterministic, opinionated formatting to all your code. This frees you up to think about bigger problems, like the larger design of the software and the business needs you’re trying to address.

³

You can find “PEP 8—Style Guide for Python Code” on the Python website: www.python.org/dev/peps/pep-0008/.

11.4.2. Language features are patterns

Design patterns are traditionally discussed in terms of objects and the interactions between them. But there are also common patterns in the way certain ideas are expressed in Python syntax. Things that are frequently done a certain way in Python because they’re elegant, short, clear, or readable are called “Pythonic.” These patterns can be just as important as design patterns to someone trying to understand your code.

Some patterns in Python involve using data types inherent to the situation, like using a dict to map keys to values. Some patterns involve using list comprehensions or ternary operators to reduce multiline statements when something is short and clear enough to do so. Knowing what’s available to you and when to reach for each pattern is important. Knowing when not to reach for them is important too.

The Zen of Python provides a good set of general principles for writing Python code.

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one--and preferably only one--obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea--let’s do more of those!

If you view these guidelines as a light rubric, you can turn a critical eye to areas of your code that rub you the wrong way or feel funny. Seeing a bit of ugly syntax, understanding what it’s trying to do, and searching the web for “best way to do X in Python” can generate some alternative ideas. Another tactic I’ve used to learn tips and tricks is following prominent users of Python—such as Python core developers—on Twitter. You can often find information this way that you didn’t know you needed.

For a comprehensive guide to the language, books like The Quick Python Book, by Daryl Harms and Kenneth McDonald (Manning, 1999); The Hitchhiker’s Guide to Python, by Kenneth Reitz and Tanya Schlusser (O’Reilly, 2016; https://docs.python-guide.org/); and Python Cookbook, by David Ascher and Alex Martelli (O’Reilly, 2002), can help immerse you in the language.

11.4.3. Terms to start with

You can start with the following terms as you begin your research into examples, patterns, and guidelines for Pythonic code:

• Pythonic code
  • Pythonic way to do X
• Idiomatic Python
• Python anti-patterns
• Python linters

11.5. Where you’ve been

As an author, I can’t predict what motivated you to pick up this book or how much experience you had when you did. If you’re still reading this, though, I do have a better picture of where you are now. You’re your own worst critic, so here at the end, it’s important to recapitulate everything you’ve learned. As you see it all laid out before you, keep a couple of things in mind:

• Software development isn’t any one thing, but a myriad of practices that coalesce into software in the end.
• Balancing all these practices will be an ongoing challenge, and some practices will ebb and flow as you focus on improving others.
• None of this is an exact science. Take statements claiming that something is “the one true way” with a big grain of salt.
• You can apply the principles you’ve learned in this book to most any language, framework, or problem. Python is great, but don’t box yourself in.

11.5.1. There and back again: A developer’s tale

You jumped right into the idea of software design in chapter 1. Understanding that software can be an intentional, thoughtful process laid the foundation for all the following chapters. You will occasionally be hard-pressed to find time for up-front design because of deadlines and the like, but try to find your center as often as possible so that you can be deliberate about the software you build. The outcomes remain the most important goal, but design will help you continue achieving outcomes as smoothly as possible.

Chapter 2 introduced the foundational practice of separating concerns. Most modern programming languages encourage the use of functions, methods, classes, and modules, and for good reason. Breaking your software down into its constituent parts helps reduce cognitive load as well as improve the maintainability of code. Concerns can be separated at the lowest levels of code, all the way up to the broader architecture of the software.

Building on Python’s structures for separation of concerns, you learned to use them for abstraction and encapsulation in chapter 3. Freeing yourself and other developers from the minutiae of a particular task unless they’re interested in knowing more provides welcome relief. Exposing only critical details to other areas of software also reduces integration points and the likelihood of breaking code for your consumers.

Moving into more concrete territory, you learned about designing for performance in chapter 4. You saw some of the data structures Python provides and in what situations they’re useful. You also learned about some of the tools for quantitatively measuring the performance of your software. Metrics measured trump speculation about what’s fastest.

Where chapter 4 showed you how to test whether programs are efficient, chapter 5 focused on testing whether programs are correct. Functional testing helps you verify that you’re building what you mean to build. You learned how functional tests are structured and how to write tests using Python tools. Functional testing patterns are quite similar across languages and frameworks, so you can take this information with you most anywhere.

Armed with some of the underpinnings of design, the next part of the book took you on a practical journey by building the Bark application. Through the course of this journey, you reached a number of milestones:

• You built a multitier architecture to support separate presentation, business logic, and persistence layers.
• You opened Bark up to extension to more easily add new functionality, and then added a new feature to import stars from GitHub as bookmarks.
• You used interfaces and the command pattern to further reduce the work needed to add or change features.
• You loosened the coupling between different areas of Bark, opening it up to new possibilities, like making a mobile or web app.

A bookmarking tool isn’t flashy, but you learned some flashy techniques in building it. Applying the body of knowledge you’ve gained to your future projects for real tasks is bound to give you similarly effective results. You can get practice with any new concepts you learn by applying them to Bark as well. You may choose to add features, improve the existing code, or write tests for it. The sky’s the limit!

11.5.2. Signing off

You’ve graduated from this book. It’s been a pleasure teaching you, and I hope to hear tales of your journey as you go on to bigger and better things with software. Celebrate the wins, learn from the hurdles, and develop with heart.

Happy coding!

Summary

• Learning is not a passive process. Make a plan that works for you, write it down or map it out, and track your progress. This can generate more ideas or next steps to help keep you motivated and curious.
• Try to identify common patterns and approaches to problems. As you encounter those same problems, try a few different approaches early on to see which work most smoothly. Patterns are tools, and they should enhance your work rather than hinder it.
• Feel at home in your language. You don’t need to pick it up all at once, but keep a curious mindset and ask often if there’s a more idiomatic way to express a thought in code.
• You’ve come a long way from the start of this book, so use this time to reflect and take a break.