Choosing a single source of truth

Even the smallest companies use a number of different tools for the same set of tasks. Your company might use both GSuite and Office 365, with some departments preferring one product over another. Even within your own engineering organization, you may have documents speckled across GSuite, GitHub, and an internal wiki. As someone searching for information about a product feature or in-flight project, having to search half a dozen different platforms for disjointed pieces of information is aggravating. It can be even more frustrating when there’s pertinent information in more than one location, and the information doesn’t agree.

When you kick off your the refactor, choose a platform your team enjoys using to collect all documentation related to the project. Because you’ll be regularly creating new documents and updating existing ones, you’ll want to choose the solution that has all of your favorite bells and whistles. If you’re annoyed every time you need to add something new, you’ll be less likely to do it, and the documentation will fall out of date.

Within your chosen platform, create a directory to house all pertinent documentation; this will serve as your single source of truth. Documentation can include technical design specifications, the execution plan you developed in Chapter 4, meeting notes, postmortems, etc. Wherever other engineers look for documentation, either link to your directory, or better yet, link to a specific document within it. If your colleagues have muscle memory around searching GitHub for technical documentation but you prefer writing in Notion, create an entry for your documentation in GitHub and link directly to your Notion entry. This way, not only will your documentation be easy to find, you’ll be certain that there aren’t any outdated copies floating around.

As your team generates documentation throughout the execution of the project, make sure that it all lands in your project directory (with updated external links from other widely used document sources).

Setting expectations

Next, you’ll want to set expectations with external stakeholders. Many of these stakeholders will regularly check in with you, “polling” for new information. Unfortunately, this model can become quite bothersome the more stakeholders you have. If you or your manager receive an email or message every time someone in upper management has a question about how the refactor is progressing, before long, you’ll end up spending quite a bit of time answering these requests. On the other hand, there may be some stakeholders that you wish would check in periodically but unfortunately do not. When this is the case, your team must “push” information out. Consistently needing to proactively propagate information out to numerous stakeholders can get irritating, particularly if the receiving party doesn’t acknowledge that they’ve read the information you provided.

Instead of either answering each request or reaching out to each stakeholder individually, you should spend some time determining how you intend to communicate progress and setting up expectations with your stakeholders early as to where and with what frequency they should expect these communications. When stakeholders break from the patterns you’ve established (e.g. you get a ping from your skip-level), instead of providing the information directly, simply reply with a gentle reminder of where they can find what they need.

When you kick-off the refactor, take some time to draft a rough communication plan. This plan should include information about the following:

• Where stakeholders can find information about the current stage of the refactor. There are a number of places where you can make this information easily accesible to external parties. If your team uses Slack, you can create a channel to house conversations pertinent to the refactor and set the channel’s topic to a short description of the current stage of the project. At the end of the week, post a weekly round-up message detailing the progress made over the last few days. (If you hold weekly sync meetings, you can draft this message immediately afterwards and link to your meeting notes.) If your team uses JIRA, provide a link to the project board. For stakeholders who need regular, high-level updates, consider adding a summary field which the team updates weekly on the top-level project.

• Where stakeholders can find a high-level project timeline. You can include a high-level project timeline at the root of your project documentation directory, directly within the communication plan itself, or as a subsection to your execution plan. Make sure to keep this timeline updated if any dates end up shifting as the project progresses.

• Where engineers can expect to find technical information about the refactor. Here, you can link to the directory where your team intends to draft documentation related to the refactor. Provide a short summary of the kinds of documents the team plans to aggregate there.

• Where stakeholders can ask questions. There are going to be instances where individuals across the company are either not going to be able to find the information they need in the provided resources, or prefer asking the question directly rather than locate the information for themselves. When that happens, you’ll want to make sure that they know where to go. If your team uses Discord, either direct them to the project channel, or set up a channel exclusively for questions. If your team relies on email and your team has an email group, have them send an email to the team as a whole rather than an individual. If your team is cross-functional, set up an email group for everyone involved and direct questions to that group.

• When impacted teams should expect to hear from you. When coordinating with teams that risk being impacted by the refactor, you want to maintain a high level of transparency. You want to make sure that no one on these teams is surprised or set back by the work your group is doing. To make sure that everyone is on the same page, provide a list of guidelines you intend to follow when interacting with other teams’ code. This could include tagging one or more individuals from that team for code review when modifying code for which their team is responsible or attending their stand-up to provide updates on the refactor when pertinent to their team.

Progress announcements

Progress announcements are not only important in order to let everyone know that you’ve completed another milestone (and unlocked any number of benefits as a result), they are also crucial in continuing to make your team feel productive and boost their morale. Large-scale refactors can feel daunting for teams, even for teams accustomed to working on lengthy projects. Celebrating each milestone as it wraps up helps everyone feel a sense of achievement throughout the duration of the project.

However your company announces the launch of new features, whether that’s a department-wide email or a message in a Slack channel, inquire about providing important progress updates for the refactor. Your team will get important recognition for their hard work and demonstrate to a wide audience that refactoring is a valued engineering investment.

Execution plan

In Chapter 4, we learned how to draft an effective execution plan for our large-scale refactor. We can go beyond using this plan as a simple road map and use it as a place to documenting our work throughout the project’s progression. Take the original execution plan and make a copy. The original version should remain relatively untouched beyond light updates to estimates and milestone metrics as necessary. The copy will serve as a “living” version of the original document and should be progressively updated as the project develops. (Enabling version history will give you the ability to easily go back in time and compare your initial values with your latest updates.) This could include anything from strange bugs encountered, unexpected edge-cases uncovered, or diversions in the plan. This second version of your original execution plan should give any stakeholders a much more nuanced view into your progress and help your team keep better track of the work they’ve achieved to date.

Take, for instance, our example from Chapter 4 where the software team at Smart DNA was tasked with migrating all Python 2.6 environments to Python 2.7. We’ve copied over the first milestone of the team’s execution plan as follows:

• Create a single requirements.txt file.

  • Metric: Number of distinct lists of dependencies; Start: 3; Goal: 1

  • Estimate: 2-3 weeks

  • Subtasks:

    • Enumerate all packages used across each of the repositories.

    • Audit all packages and narrow-down list to only required packages with corresponding versions.

    • Identify which version each package should be upgraded to in Python 2.7.

As the software team begins making progress on the migration, they might start filling in the copy of their original plan with more context on their findings. We can see some of those additional details in the plan that follows:

• Create a single requirements.txt file.

  • Metric: Number of distinct lists of dependencies; Start: 3; Goal: 1

  • Estimate: 2-3 weeks

  • Subtasks:

    • Enumerate all packages used across each of the repositories. When we started combing through all of the packages used by the first of the three repositories, we were surprised that the code relied on 6 additional dependencies that weren’t explicitly listed the respective requirements.txt file. The researchers were able to provide an updated list for the first repository, as well as the 10 other dependencies missing from the requirements.txt files for the other two repos.

    • Audit all packages and narrow-down list to only required packages with corresponding versions. Thankfully, 80% of the packages used by the three repos were the same. Of that set, only 8 of those packages had different versions that needed to be reconciled.

    • Identify which version each package should be upgraded to in Python 2.7. This was tricky for 7 of the packages in the final, combined set. For these packages, their 2.7-compatible versions deprecated a number of APIs and features that the researchers actively used in two of the three repos. We worked with the research team to gradually migrate away from using these deprecated features before continuing with the refactor.

Updating the execution plan as you go means that others can reference it throughout the project’s lifetime to get more context on the specific work the team is doing during each of its many stages. Any subject matter experts joining the project at a later milestone (or any new teammates being onboarded partway through the refactor) can ramp on up on everything the team’s worked on to date just by reading through the execution plan. If you’re anything like me, sometimes you forget why you made a certain decision several months ago; by keeping a verbose account of everything you’ve encountered and the conclusions you’ve reached along the way, you’re able to easily go back and remind yourself of precisely what happened and why.

A detailed account of the team’s experience can also be helpful for engineers and managers referencing the refactor well after it’s completed. Engineers seeking to understand how the codebase has evolved over time may want to read your detailed plan. For the engineers involved with your refactor seeking a promotion, having concrete documentation pointing to the highly technical problems they solved at each step can be incredibly valuable. Elsewhere at the company, engineers looking to kick off their own large-scale refactor might look to your documentation for an example of how to executed a substantial refactor successfully.

Seeking feeback from senior engineers

All of us seek out advice from peers and experienced colleagues when solving difficult problems. While we might be eager to request feedback from senior engineering leaders (and benefit greatly from it), getting and retaining their attention can be notably difficult. Whether they’ve been engaged with the refactor from day one as subject matter experts (see Chapter 6), or are just getting up to speed, they’ll likely be slower to respond to your inquiries simply because they are unusually busy with many responsibilities across a multitude of projects. Ideally, if you are able to communicate your expectations appropriately, none of these individuals should become bottlenecks.

When soliciting feedback from these senior engineer leaders, we must first decide the scope of the feedback we’re looking for. This is helpful for two main reasons. First, explicitly defining which aspects of the problem or solution we want our colleague to evaluate ensures that we won’t get unexpected, frustrating feedback on pieces we’ve already nailed down. Second, they’ll be able to immediately focus on just the essential pieces, saving them a great deal of time and energy they would have otherwise spent assessing a much greater problem.

Next, we have to determine how crucial their feedback is to the momentum of the project; that is, can you continue to make progress without their input? If you believe you can continue to make progress without their opinion, be clear explicit about it. This way, the engineer can properly prioritize giving you the feedback you need against other similar requests they might be juggling from other engineers across the company. If you believe their input is required for your team to continue making progress, letting them know that they are now a blocker should give them adequate urgency to get back to you quickly. Regardless of the urgency, you should set some clear expectations for when you need their feedback by so that no one is left twiddling their thumbs.

We also need to consider how crucial the senior engineer leader believes their own feedback is to the momentum of the project. If you both agree that their input is not a blocker, great! But if there’s a chance that they’ll be surprised and disgruntled if you move forward without taking their opinion into consideration, you need to be aware of it so that everyone’s expectations are properly aligned.

To illustrate this in action, let’s say you’re working on a prototype for a new library you’re building as part of a large refactor. Your prototype defines some basic interfaces, with a handful of temporary, incomplete implementations. You put up your changes for code review, complete with a short description and links to a design document developed by your team. You want some feedback from a senior engineer, so you tag them and a few other teammates for review. Unfortunately, you forget to tell the senior engineer that you’re looking for feedback on the interfaces (not the implementations) and are hoping to merge these changes within the next week.

A few days pass with comments from your teammates but nothing from the senior engineer. You send them a message asking if they’ve had a chance to take a look at the code review. They assure you that they saw the request and that they intend to get to it by the end of the week. After some back and forth with your teammates, you decide to merge the prototype and continue iterating on it in subsequent code reviews.

A day later, the senior engineer opens up your code review and begins to read through it. They immediately begin commenting on the implementation details, becoming increasingly alarmed as they realize that the code has already been merged. Now everyone’s irritated: you’re irritated that the senior engineer took too long to review your changes and ultimately focused on the wrong aspect of the code; they’re irritated that they left comments on what turned out to be temporary code and that you’ve merged your changes without waiting for their input. All of the disappointment and miscommunication could have been avoided had the right expectations been set from the start.