This chapter aims to give you a leg up on the mechanics of life away from your IDE. We’ll cover tools for organizing projects, time management, record-keeping, meetings, and interviewing candidates. None of these skills are complicated in principle, but all are invaluable for getting results on real teams and showing your teammates that you’re a mature professional.
Project Management
Project management is organizing technical work to deliver complex projects: planning, tracking, and communication to divide work into coherent pieces, allocate the right resources, do the right things at the right times, remember the loose ends, and unblock engineers (including ourselves).
This section will argue that you need to be a good project manager and offer a primer on how to get there.
Motivation
At every scale except the tiny bugfix, engineering work demands organization. Project costs must be estimated and weighed against returns; work must divided to map onto engineers; schedules must be communicated to stakeholders; key pieces must be tracked so what ships is complete.
One-person projects (which are uncommon!) require engineers to own every aspect of a project. At the other extreme, 50-person projects may have dedicated project managers, but engineers still need to manage their own work and understand how it interacts with other teams’. In between, engineers need to rise to the task of leading small teams themselves. At every level, engineers must own their own timelines, break their own work into individual tasks/diffs, and cross all their Ts to ship.
Therefore, successful engineers are usually excellent project managers and successful engineering managers doubly so. You should be capable of managing a project as well as a professional project manager, just as the best technical project managers can dig into technology when they have to. And, since we hold ourselves accountable for the success of large projects, we need to be ready to step into project management duties at any time to shepherd a team or area to success. Even now, for the past year, I haven’t had a TPM1 for a ten-person slice of a larger project, so I’ve been doing it myself.
The Foundation: A List and a Graph
Keeping a list of all outstanding tasks—a spreadsheet, text file, JIRA board, whatever.
Understanding the dependency graph of those tasks—a flowchart or a DAG2 where an edge from A to B says that B can’t be done until A is.
These methods establish what needs to be done and how much can be done in parallel; they inform prioritization by showing which project stalls are critical (because they block other work) and which are harmless; they ensure that no known work is forgotten before you release.3 Armed with those constructs, you can estimate timelines, ask for resources, run meetings, and send updates to stakeholders.
Example task list
The graph may be less obvious. It serves two purposes.
First, it informs prioritization. Engineers always want to start on the wrong piece of a project—on the most fun piece, on the piece they know best (or least), or on just about anything other than what advances the project the most. When you look at a dependency graph, the nodes with the most outbound edges block most other work—you can see which API needs to be defined for three other tasks to start or realize that the missing tooling is blocking five other teams, while the fun performance optimization can easily be deferred and done anytime. The discipline to do that analysis and start with what matters most is enough to make you an above-average project manager.
Second, it lets you schedule, in the senses both of assigning resources (like a CPU scheduler) and predicting timelines. If your graph is basically a list—each task depends on the previous one and nothing else—then no parallelism is possible. You’ll do each piece and feed it into the next, and maybe you’ll do it all yourself; the time it takes will be the sum of the individual pieces. If there are no edges in the graph except to the “done” box, then each piece is completely independent and you may have infinite parallelism; the total time might be the max of the times of the individual pieces (modulo the complexities of estimation, discussed later in this chapter).
Example dependency graph
Keeping Stakeholders Informed
When you manage a consequential project, people want to know how it’s going—either because they need to use the results (e.g., they need to use your API) or because your code is their business commitment (e.g., they’ve promised the feature to a customer or executive).
- 1.
Timeline
- 2.
What’s done
- 3.
What’s not done
- 4.
Implications for downstream systems
Managers and project managers usually care more than anything about #1. Technical partners will care about all of the above.
For most projects, we should aim for weekly updates. That cadence establishes a comfortable rhythm where you show a commitment to transparency, and people never wait long enough to start feeling that they’re in the dark. For projects under critical pressure, for example, in day-for-day slip against a deadline, briefer daily updates may be appropriate.
We usually update stakeholders one of two ways: email or meetings. Email is the preferred method for large projects, because it can be consumed asynchronously and partially (especially when you have a good tl;dr—see Chapter 13); large status update meetings are strongly discouraged (by me, though they’re widely used), because they chiefly consist of tens of highly paid people sitting around, bored, listening only to their own worried thoughts about how they’ll word their own update.
For emails, use the project status update email template (Chapter 13). To provide an update in a status meeting, you can use the same template, but strive to be even more brief!
Keeping Close Collaborators Informed
When you’re working closely with other engineers, you should communicate far faster and more loosely then with external stakeholders or a big team. I ping close collaborators freely with updates on minor progress and hiccups, ideas I’m having, questions, thanks for their progress, etc. and above all problems that might waste their time—trying to avoid even one second lost to tactical miscommunication and showing them that I value their time, that I want to deliver for them.
quick fyi—i just landed the build changes, you should be able to adopt the new rule now, please let me know if you have any problems.
hi anant, sorry i didn’t get to your code review earlier. i just added some comments. i’ll treat it as my top priority for the rest of the day, so feel free to ping me if you update the diff.
hey evelyn, I just found a bug in my script. it might not rebuild the coordinator config when you do a dev build. i’m working on a fix right now.
yo, I just saw that you landed a fix to the reconnect problem we were hitting—that’s awesome, thanks so much.
[to a chat room] hey everybody, if you’re seeing failures when turning up a dev cluster, it’s probably the bug Jeff is working on. I just landed a fix to the workaround script and updated this channel header with a link to the workaround runbook. sorry for the inconvenience, we’ll have a proper fix asap.
These are streaming up-to-the-minute updates, intended to keep people unblocked and productive; these kinds of proactive comms keep collaborators productive and happy, as well as encouraging a broader culture of support.
Schedules and Estimation
Four Laws
- 1.
They’re usually artificial, but you still need to care: Schedules and deadlines sometimes come from real-world constraints, like regulatory deadlines. Much more often, however, they are artificial, born of a desire for predictability in managing a company. Shareholders want to make sound investments with their dollars, which requires understanding costs and returns; therefore, CEOs must understand returns on engineering investments (e.g., $100,000,000 a year for their engineering department); therefore, CTOs must understand the returns on teams and projects (e.g., $1,000,000 for a substantial project or five engineers for a year); therefore, your manager sure cares about being able to quote a timeline and stick to it. Finally, you should care, because you’re accountable to people who look bad if you don’t deliver. It can be tempting to feel contempt for artificial deadlines, but disrespecting them won’t get you anywhere.
- 2.
Estimation is the hardest problem in software: Most software development is fundamentally unpredictable, dominated by the struggle to reveal the true complexity of problems that have not been (in every detail) solved before. Not only is the work itself unpredictable, but your schedule is as well—normal engineering involves frequent interruptions (e.g., a sudden customer problem) that will take up your time and reduce your focus. You will learn quite quickly that you and everyone else are quite bad at estimating how long even a bugfix will take.
- 3.
Therefore, pad your estimates by 2x: I’ve never seen a system for software estimation that was truly accurate; the more complex the system, the less trustworthy. However, padding by 2x seems to be surprisingly effective. Sketch the major pieces of your work, make your best wild guess at how long each will take, add them up, and multiply by 2, and you might come close. As a special case of the principles of doing at least as well as you promise your stakeholders, it’s far better to promise late and deliver early than vice versa. Managers and product managers will try desperately to get you to shave time off your estimate; do your darnedest to resist. The 2x multiplier comes partly from inefficiency—we spend less time coding per day than we think—and partly from Simon’s Law,4 which says that productionizing code takes about as long as writing it.
- 4.
Overcommunicate your status: You’re going to slip many schedules in your life (see rule #2). The right time to communicate that you’re going to slip is the very first microsecond you’re aware of it; if you’ve chickened out on that first moment, then the second best time is right now. The later the news, the more embarrassing it will be to you and everyone who depends on you, but a prompt, clear notification that a schedule is changing can be a sign of professionalism appreciated by one and all. For example:
Hi Folks, Today, we realized that before we can turn on the new session cache, we need a way to shoot down sessions on account deletion. We already have shootdown for password change, but hadn’t previously considered the account deletion case. We estimate that this will add a week to the total time to delivery, meaning that our new estimated date for go-live is March 15th. Please reach out with any concerns. Cheers, Yun |
If your manager is going to be angry, that is certainly their professional failing rather than yours, but telling them later is guaranteed to make it worse, not better. If you’re on track, you’ll still do well to send crisp weekly email summaries of your progress and immediate goals.
Schedules and Estimation: The Basic Algorithm
Now that we’ve established that careful scheduling is futile and necessary, let’s talk about the nuts and bolts of doing it; fair warning that this section won’t be rocket science.
- 1.
Sit down with a pen and paper (it clarifies your thinking better than a laptop).
- 2.
List the major pieces of your work; draw your dependency graph.
- 3.
Estimate the time for each piece.
- 4.
Sum the estimated time for each work item that cannot or will not be done in parallel.
- 5.
Add time for testing; make it much longer than you expect, because you are guaranteed to underestimate.
- 6.
Double the total.
- 7.
Resist requests to reduce that estimate.
This method is simple to the point of obviousness, but I haven’t found a better one.
Leading a Project
At any scale, you depend on others’ work; as you lead larger projects, you’ll depend much more on others’ code than on your own. This section will discuss how to shepherd others to a successful delivery of a project.
If you remember only one principle of project leadership, remember that as project lead, your job is to deliver the project, full stop. There’s no good excuse for not delivering, and therefore, you can never accept being blocked. You advocate for the resources, you weigh the quality trade-offs, you estimate the timeline, and you persuade and coerce the help you need; when things seem bad, you do the exercise of thinking as if you were your boss,5 asking yourself which of your assumptions about resources or requirements could be thrown out.
- 1.
Bootstrap your project understanding by researching and meeting with stakeholders and participants.
- 2.
Draw your actual dependency graph and estimate timeline; write an accompanying concise project plan. Ask participants to estimate and commit to their own timelines.
- 3.
Communicate this plan and rough timelines (conservatively estimated) to participants.
- 4.
Set up your tracking, as discussed above.
- 5.Repeat weekly:
- a.
Communicate: A fair start is a weekly meeting, led by you and ruthlessly kept short, where people give updates and you discuss areas of concern. You may send a weekly status update up the chain or to other stakeholders.
- b.
Update your tracking and dependency graph.
- c.
Follow-up 1:1 to go deeper on areas of concern or uncertainty.
- d.
Review progress against your timelines, and unblock the pieces that get blocked, whether by nagging (see the next section), advising, or doing things yourself. Update stakeholders immediately if timelines change.
- e.
Execute on your personal piece of the project.
- a.
The Fine Art of Nagging
Follow-up is an essential part of project management—making sure that others really do the things we need them to do, from reviewing changes to building distributed systems out of whole cloth. This section will describe how to nag people to do things you need them to do.
What to Say
As in every other chapter of this book, your primary weapons are honesty and empathy. We start with a strong grasp of the project plan and frankness about what’s needed, because we won’t get what we need by asking for something else. We encourage, charm, thank, and come through for others so they feel glad to work with us and help us or, at least, less put upon (see Chapter 6). And we apply the minimum emotional force necessary to communicate our needs. The secret ingredient for perfect nagging, though, is acknowledging others’ struggles—that they have competing priorities and that your ask may be difficult.
When our partners have committed to a schedule and we have no reason to suspect a problem, our question can be very simple: Are we on track?
Hey Kara, just checking in—are we still on track for rollout in two weeks? Is there anything I can help with?
If the answer is “Yes,” we can probably get right back to our own work. Things get much more interesting if you know or suspect that you are not on track; everyone involved is already tense, and we have to find our delicacy. Here’s a question I don’t like:
Hey Kara, have you fixed that uncaught issue yet? It’s blocking the rollout.
Our implication is instantly that Kara should have fixed the bug—that we expect her to, that we’re suspicious that she may not have, and that we’ll look down on her if she hasn’t. Why ask that way when we can ask as a friend?
Hey Kara, have you been able to look at that uncaught bug? I know you were also working on the fallback mechanism, so not sure if you’ve had a chance, but we do need it for the rollout.
To me, this is all the difference in the world—we’re signaling that we care about this bug, reminding Kara that it exists and that she should solve it, and communicating its priority—but we’re not implying any judgment. On the contrary, we’re acknowledging that she’s doing important work and that she may have a good reason for not yet giving us what we need. Here’s another empathetic example:
Hey Youssef. I wanted to check in with you about your progress on the secrets migration. How are things going? I know your team’s usage is a bit unusually complicated—do you folks have the resources to get this done by the end of the month, or would some help from the secrets side be useful?
Here, we’re acknowledging that an ask we’ve made may be particularly tricky—that a team may have a good reason for being behind schedule. However, at the same time, we’re sending the message that the deadline matters and we want to work with them to hit it.
In short, we’ve nagged well—we’ve said what we need while encouraging our colleague.
When Gentle Requests Don’t Work
Increasing the intensity still means talking person to person; it just means expressing urgency.
Hey Vijay, just following up again about the uncaught bug. Sorry to keep bugging you, but this has now become a major problem for the on-call rotation; would you be able to look at it today?
This is the natural evolution of the strategy from the previous section. We prefer it over involving other people, because it’s more personal and involves less risk of embarrassment for our colleague.
If that fails, we escalate. Asking a project manager to follow-up is the next incremental escalation; it’s milder than going to an engineering manager, because project managers’ essential function is to understand outstanding work and ensure that it happens; their follow-up doesn’t involve an implication of negligence, unlike reaching out to someone’s manager. If a project manager isn’t available, we go to our manager or our colleague’s (who may be the same). Our question should always emphasize the task, never the negligence of our colleague.
Hey Sandra, I just want to put this task on your radar; it’s been lingering for a while, and I think we need to make sure someone follows-up on it, because it’s impacting on-call with several alerts per shift.
I would usually do this with a direct outreach, not with an email to both the engineer and their manager. Reaching out to both at once is the last step of escalation, because emailing an engineer and their manager is always accusation of negligence. Sometimes we really need an attitude adjustment from our colleagues, and that’s how we can do it. In either case, the admonishment above applies—focus on the task and its impact, not on the negligence.
Advanced Unblocking: Doing It Yourself
I’ll mention one advanced technique, not to be used too lightly. When someone fails to follow-up, you can just do their job for them. This might solve your problem, and it might set an inspiring example of vigor and ownership. It can also send a positive message—if your partners really wanted to help but simply couldn’t find cycles, and you do it the same way they would, as a partner, you’re a friend.
Hi Francisco. Since I know your team is slammed, I thought I might take a shot at this project to unblock the bigger effort. I’ll check in with you along the way and make sure your team approves the design and code. What do you think?
On the other hand, if you go rogue and build an alternative in secret, you’ve shamed them, which you should never do lightly, but sometimes it’s necessary.
I once saw a team constantly promise the moon and deliver nothing on a key component for a full year (a debugging tool)—its lack was blocking a critical project. An engineer I knew built a fully functional hacked-up alternative in a single week. It was a scandal; the other team’s manager was eventually censured, tears were shed, and it took countless meetings to smooth over the tension between the teams. On the other hand, the project got unblocked. I guess you see the trade-off.
How Often to Follow-up
There’s no easy recipe for scheduling your follow-up. All I can say is, err toward less when you can, because it’s stressful to be asked for status. For a critical production issue, every 120 seconds is appropriate; for a broken tool that’s blocking engineers, maybe once an hour; for a tricky bug blocking a release, once or twice a day; and for an important project that needs to land months in the future, once every week or two.
When I’ve settled on a wait time, I set a timed reminder in my to-do list app to ensure I won’t forget (“two days from today: ask Kara about the schema change”). Following-up reliably, like many feats of quotidian organization, is more impressive than you’d think.
Paying for Information
Estimation is so hard because software projects are filled with unknowns; part of the beauty of our craft is that there are such spectacular ranges of methods and tradeoffs available to us on almost every task. Therefore, early in a project, information is our most precious commodity. We need to swiftly illuminate the map and understand where our challenges will actually lie; therefore, we prioritize tasks that yield information and reduce uncertainty, and we may make quality trade-offs to race past pieces we’re confident about. Generally, we ask: What puts this project at risk, and how can we reduce that risk as quickly as possible?
“Prototyping,” that is, building fast and at low quality, is a tool for that purpose—when we need information, we buy it by building something optimized for answering our most important questions without agonizing over the tricky error handling, the perfect abstraction, complete observability, etc.
Much is made of the value of always building “throwaway” prototypes, and I think that’s too extreme—I’ve seen people build things to throw away when they could have built right the first time. Prototyping isn’t its own reward but a way to pay a favorable price for a lot of information; we should neither fear throwing away code nor resist, when possible, paying a little more for code we can reuse.
Effective Meetings
Meetings can be powerful tools to reach consensus and keep projects on track; they can also be, and usually are, time-wasting nightmares. Good meetings have clear goals agreed on by one and all; stay focused on achieving those goals, and minimize the time to achieve them, and they produce clear plans for action after the fact. Bad meetings do pretty much anything other than that, but especially, they meander without a goal or end without a plan.
Running good meetings is a game changer for making progress on controversial or cross-functional issues. In case that alone doesn’t motivate you to read on, I’ll note that, like many neglected soft skills among engineers, you’re apt to surprise and delight both peers and bosses if you can run a solid meeting; it’s well worth your time to study up and find opportunities to practice.
Throughout: An owner who takes overall responsibility for making a meeting productive
Before: A clear agenda, with context, sent out beforehand
During: Punctuality, minimal distraction, and above all, steady focus on reaching conclusions as quickly as possible
After: Clear meeting notes sent afterward to confirm the decisions of the meeting
Let’s explore each in detail.
Throughout: An Owner
Meetings tend toward entropy—all of our efforts are toward turning a naturally chaotic system of different people with different context, goals, and responsibilities, into a productive, organized process. Just like any other task you want done, someone needs to own the fight against meeting entropy—to make sure that the preparation, conversation, and follow-up happen smoothly.
The meeting owner takes on that responsibility. It’s seldom a formal role, but you’ll find that in productive meetings, someone steps up to take ownership. Most importantly, when you call a meeting to achieve one of your goals, you should personally guarantee the other key ingredients for a successful meeting—send the agenda, shepherd an efficient conversation, and write the meeting notes. Sometimes, one person may call a meeting but (explicitly!) delegate ownership to another attendee.
Before: Agenda
Before any meeting, the owner should write up the meeting’s goals and a list of steps to achieve them; the write-up should link to relevant context. That agenda can be included in the meeting invite, sent by email, or both. This up-front communication lets attendees prepare (read, think about the issue), but, perhaps more importantly, it focuses them on the goals—if they know what you need, they can help achieve it. Perhaps most importantly, it requires the meeting owner (you) to think through the goals and method!
Suppose this is a meeting about an infrastructure decision. Here’s what an agenda, included in a meeting invitation, might look like:
Calendar Invitation Title: Meeting re. Database Choice for Next Gen Backend
Calendar Invitation Body:
The goal of this meeting is to make a plan for deciding on the database for our next generation backend.
- How to evaluate performance.
- Sketch budget and possible tradeoffs.
- Confirming level of support from Infrastructure team.
- Back of envelope analysis of impact on system design.
- Owners for follow-up in each area.
Manuel will open the meeting with a brief review of scalability issues with the current system and the known requirements for next generation.
Pooja will then give an overview of the infra team’s initial investigation and the short list of options to consider.
Finally, we’ll have open discussion on the preceding points in that order.
This invite explains the goals of the meeting in some detail. It explicitly calls out owners for providing context in the meeting and links to some detailed background for attendees to prepare themselves.
You may ask: What about long status meetings where everyone gives their report and others have a chance to listen and give feedback? I’m not a fan. However, sometimes the pure information sharing can be valuable to catch problems on complex projects; my advice to you is, in that case, push hard to make everyone move fast and to make everyone else pay attention—try to get the information truly shared and everyone back on task.
During: Punctuality
Lateness wastes others’ time and sets a tone of poor focus. Set a tone of businesslike precision, with the expectation that you’ll start on time and get done early, and everyone will improve their focus and efficiency. You don’t need to wait for everyone to arrive to start; you have quorum when all key decision-makers and technical experts have arrived.
During: Start with Agenda Review
Once you reach quorum, the meeting owner should start by reviewing the agenda. Some attendees will have failed to read it; others will have forgotten; everyone will benefit from a brief, clear refresher to nudge them in the right direction.
Thanks for coming, everyone! As you know, we’ve decided to move off of SimpleDB for our next gen backend, and we need to decide how our teams will work together to make the choice.
Our goals in this meeting are to confirm a plan for choosing a database, especially identifying an owner and PoCs for driving the decision, but also running through key concerns for performance testing, budget trade-offs, and ownership/support.
Manuel will start by reviewing the current system’s problems and requirements we know about, and then Pooja will discuss the options the DB team has come up with in their initial investigation. After that we’ll talk about performance, budget, support, and next steps/owners.
Let’s get started—Manuel?
Now you’re ready to get down to business.
During: Focus on Reaching Conclusions
Every meeting should achieve actionable conclusions, reviewed succinctly at the end of the meeting—otherwise, how is anyone better off? If a meeting doesn’t seem to be moving toward actionable conclusions, you should try to push it there—notably, any discussion that isn’t helping should be swiftly deferred (or just dropped); your colleagues are well-paid professionals, and every minute of their time is money spent. You’ll find that when you successfully refocus a meeting, your peers will be surprisingly impressed and appreciative; for a long time, this was my only skill, and I seemed to get by better than you’d think. You can say things like:
What would help us reach a conclusion here?
Could we table this for now and focus on <the main question>?
This is definitely an important question, but since we’re short on time, maybe we should move on to the next topic and circle back later?
I’m not sure if we can decide this right here—<person>, maybe you could analyze this offline and email us your assessment?
Do we have the right people to figure this out? Should we pause this conversation until we can get input from <an expert>?
During: Review Action Items
If anything needs to happen based on the discussions in a meeting, the meeting should end with a review of clear action items with clear owners. If something needs to be done and doesn’t have an owner, it will not get done.
Okay, so our action items are: Erika to send guidance to the team and Donald to start the rollout tomorrow morning. I’ll send the meeting notes.
After: Meeting Notes
When a good meeting is over, someone (agreed on at the meeting, by default the meeting owner) should usually send crisp meeting notes reviewing the purpose of the meeting, who attended, what was discussed, what was concluded/agreed on, and what the action items are (with owners). Within one team, or at a small company, formal notes may not be necessary, but when multiple teams are involved, a write-up minimizes the risk of conflicting memories of what you agreed on. Otherwise, you can hit some nasty conflict down the line, like two people each think the other is going to write a certain piece of code. Here’s an example of my typical meeting notes:
Email Subject: Meeting Notes for Database Migration Sync 2020-02-29
Email Body:
Subject: How to decide on database system for next-generation backend
Attendees: Angie, Ferras, Chris, Syrie, Song, DeMarcus, Manuel, Pooja
Notes:
- According to Jing, feedback from Infra team is that BuggyDB is fully supported, but AgamemnonDB support is still in experimentation phase.
- Concerns about replication reliability with BuggyDB are not sufficiently severe to warrant ruling it out, we’ll move forward with performance experiments.
- Action Items:
- Syrie will confirm infrastructure budget for this migration w/Donald .
- Song & Chris will take point on the performance investigation (report within a week)
- Angie will follow up with Infra team about timelines for full AgamemnonDB support.
This is a comprehensive but formal structure. If that formality seems out of place on your team (ask a colleague!) but you still want to record your conclusions, you can try something simpler:
Email Subject: Meeting Notes for Database Migration Sync 2020-02-29
Email Body:
Hi all,
Just recording our conclusions from this meeting: we decided that Song and Chris will go ahead with the performance investigation of BuggyDB, Syrie will check on the budget for the migration, and Angie will check on the AgamemnonDB support. Thanks everyone!
Cheers,
DH
A Note on Meeting Etiquette
Slouching or leaning back our chairs
Reading our phones or laptops unless strictly necessary
Interrupting
Edgy jokes
Bad-mouthing others
Complaining
Respectfully but proactively introducing ourselves
Paying attention
Waiting our turn to speak
Not dominating the conversation
Giving everyone a chance to say their piece; making space for the shy or quiet and soliciting their opinions (especially if you’re the meeting chair!)
I’ll also offer a puritanical perspective on one key part of meetings: people shouldn’t open their laptops if it can be avoided. If a meeting is so poorly run that people need to kill time, it should be improved or canceled; if your colleague is going to the trouble of speaking for your benefit, you should pay them the respect of your full attention.
Productivity and Organization
Time Management and To-Do Lists
When I started out in software (and before that, in college, high school, middle school, and elementary school), I constantly forgot things I needed to do. I’ve solved this problem with the diligent use of a to-do list app, and it’s wildly increased productivity and reliability; you should do the same.
Data must be shared between phone and desktop, so you can create to dos wherever you are (e.g., immediately after a hallway conversation).
Creating a simple to do should be frictionless, so you do it.
If you adopt only this practice, it’ll be quite beneficial, but I got even more benefit when I read a book I highly recommend, David Allen’s Getting Things Done.
Keeping contextual lists (e.g., work and home)
Regularly reviewing and prioritizing your list of tasks, making sure that you can quickly find the most urgent ones in any given context when it’s time to work; the regular grooming improves your confidence that you’re doing the right thing at any given time and therefore reduces your stress.
Read the book.
Effective Calendaring
Always include a video conference link. People work from home all the time and get held up here or there; make it easy on them.
Give your attendees edit permissions on the invite (if your system supports it), and tell them that: “You have edit rights on this invite, please feel free to move it to a more convenient time.” Busy people’s calendars will shift, and you want to make it easy for them to reschedule.
Include a crisp agenda in your meeting invite; mention each topic you plan to cover and what the goals of the meeting should be (it’s not just a chitchat session; it has a goal).
Subject: Timeline for Project Hufflepuff
- Schedule risks from DB migration
- Tax compliance implications
Goal: Confirm a deadline that can be communicated outside the org.
Defragment other people’s time; if they have a packed calendar, try to find a slot adjacent to their other meetings, and especially try to avoid creating small gaps between meetings. Most people are most effective in contiguous blocks of focused time, and you should help them create that space.
When scheduling by email, always offer a menu of acceptable times (with explicit time zones if you’re in different ones!):
Meeting next week sounds great. Would one of the following slots work?
Monday 2/1: 11 am PT or 2 pm PT
Wednesday 2/3: 1:30 pm PT
Caching Knowledge
An infinitely scrolling Google Doc, your cheat sheet, with a little section for each neat trick you figure out. This document should record things you might not otherwise know how to do—it is a durable repository of knowledge.
A text file on your computer, your command cache, with commands you’ve been using particularly recently. These may or may not be hard to figure out, but they would take time to reconstruct; just copy–paste! A text file is less durable than a Google Doc, but it’s much faster for frequent use.
There may be overlap between these two documents; the command cache is ultimately just for convenience and speed, but the cheat sheet is for saving the day when your memory doesn’t come through for you.
Scripting Everything
This advice is quoted just about everywhere, so I’ll be brief: if you type a command more than five times, you should script it, ideally down to five characters or less (e.g. gco for git checkout), either as a shell script or an alias. You’ll improve your shell scripting and save yourself endless time and wrist pain. I strongly recommend keeping your Git repository of personal shell scripts (see “Living in the Cloud”) in your path; if you go two weeks without adding a new script, you’re probably not automating enough.
Command/Alias | Definition |
|---|---|
gb | git branch |
gd | git diff |
gdc | git diff --cached |
gg | git grep |
gs | git status |
Living in the Cloud
All your dotfiles and scripts should be in a private Git repository (I use GitHub); all your documents should be safely striped to a hundred servers (I use Google Docs); getting a new laptop set up to be productive shouldn’t cost you more than a half hour, and you should never even think about “restoring from backup”—the cloud should be the truth.
Just Do Something
Perhaps the easiest way to separate effective and ineffective engineers is into those who Do Something when they encounter a problem and those who…do anything other than Something, like talking about how things could be better, waiting for someone else to solve it, or ignoring it. This virtue is sometimes called a bias to action. Managers and colleagues love people who do things rather than complaining or waiting for other people (their parents? their nanny?) to do them.
This problem arises around half the time from a misunderstanding of one’s role—staying blocked because the blocker “isn’t my job,” when in fact, your job is to get results even when you hit a roadblock.
In the other half, it comes from a lack of confidence—a paralysis induced by the discomfort and fear of not knowing exactly what to do. It’s true that responding to an unexpected challenge with vigor requires confidence in one’s ability to improvise. It’s also true that in some delicate situations, where some of the things you can do might have serious consequences, it’s best to ask for help or spend time thinking carefully—but that’s still much better than doing literally nothing, and colleagues will be much happier to help you if you show that you’ve tried your best (and ask your question clearly - see Chapter 14).
Issue | Why I Can’t Do Something | Yes You Can! Try This |
|---|---|---|
We’re blocked waiting for another team’s bugfix | They haven’t commented on the bug | Email the engineer. Email their manager. Walk over to their area. Chat someone. Call them on the phone. Ask your manager or a more senior person to escalate |
Requests are failing | I don’t know what dashboard to look at | Search for the service name on the metrics home page. Guess the name. Look at a global dashboard. Call someone and ask. @here in a chat room and ask. Page someone else |
We can’t land changes because the build is broken | I don’t know how the build works | Figure it out! Ask someone. Go to the Jenkins page and find the logs. Use git bisect, which lets you find the culprit even without semantic understanding |
My colleague blocked my diff with code review comments I disagree with | He’s too obstinate, I can’t make progress | Meet face to face with the colleague, and try to understand the feedback better. Get a second opinion from a colleague you trust, and ask if they have an idea for how to handle it tactfully. Ask your manager for advice or to intervene. Just go around them (if circumstances allow it) |
Keep a Worklog
worklog: incident commander for configuration service outage 10/14
Then I asynchronously process those to dos and put them into my doc, which is divided into a section for each quarter of the year (a common cadence of reporting). Start today.
Hiring Engineers: Interviewing from the Other Side of the Table
A Controversial Assertion
No one knows a reliable way to measure the difference between a good engineer and a bad one; therefore, no one knows how to interview engineers, and we are obliged to rely on intuition, with all its flaws. Anyone who tells you that they do know how, and many people will, is arrogant; the best you can hope for is a relatively persuasive intuition.
A good interview process would select for engineers that help the company succeed; the rigorous design of that process would require first telling the difference between high-performing engineers and low performers, then identifying qualities that predict good performance. For example, if we asked the same 5 questions of 1,000 hires, and knew which performed well and which poorly, we could try to figure out which questions gave the strongest indication of future success.
In reality, we don’t even know who is truly a high performer and who a low performer; in this domain as well, we rely fully on our intuition—managers basically just use their judgment. And even with the data we do have (i.e., internal performance reviews, flawed though they are), I personally haven’t encountered attempts to assess the predictive power of interviewing methodologies.
Therefore, you can rest easy that no one really knows how to select engineers. Still, we have to decide with the information we have (usually four to five one-hour interviews) and complaining that it can’t be done rigorously isn’t a solution, howsoever great the pleasure it gives me. What should we do?
Interview Focus Areas
If we can’t rely on science, we must rely on the best of our reasoning and intuition; in this section, I’ll talk about the qualities I look for, how I look for them, and how I put it all together into a decision.
I think many blends of skills can be successful in software, and when we’re hiring, we should keep an open mind to that variety of possibilities—a good team has room (indeed, the need) for collective excellence in every domain but doesn’t need every individual to shine in every domain. If you’ll forgive my reciting what’s now a cliché, we should resist with all our energy the desire to hire people who strongly resemble ourselves lest we both commit illegal, immoral discrimination and deny our teams the benefit of a variety of perspectives. You would do quite well to review the forms of illegal discrimination in hiring (which vary by US state and country).
This lesson was brought home for me by working with an awesome engineer I would never at that time have hired. He was for all intents and purposes impossible to communicate with, and I personally prize good communication in colleagues; he frustrated me literally every time I crossed paths with him. However, he was a consummate doer. He didn’t complain, he didn’t brag, he just went and found really tricky stuff that needed doing and did it, especially around a database system that was causing us a lot of problems; our team would have been in big trouble without him. It was humbling for me to see that he was what our team needed even though he wasn’t what I looked for (and in fact, even though I didn’t love working with him).
Coding (we write code)
Architecture (we design systems)
Communication (we almost always work with others)
Domain knowledge (knowing stuff helps you do stuff)
Organized thinking (button mashing doesn’t solve hard problems)
Attitude (dedication and respect for others pull the package together)
Coding
Why: it’s ultimately what we do, and engineers should show fluency in at least one programming language to give us confidence that they will be comfortable producing code.
How: A coding problem, ideally in a shared editor like CodePair (so you can see how they really code!). I personally do think that coding problems should, to the greatest extent possible, not be centered on one crucial little trick—there should be a smooth curve of potential outcomes, not just total success or total failure—but you can bet I don’t have science to back me up. However, I do have the intuition that coding under pressure is hard and finding a little trick doesn’t come up that often in my job.
I do prefer a coding question to include some basic data structure concepts, though it shouldn’t simply be a test of CS102 knowledge; while data structures expertise may be a shibboleth of sorts, it does seem to correlate with the accumulation of other useful knowledge in my personal experience.
I try to pay attention so I can observe the candidate’s process but, at the same time, make it explicitly clear that if they want to take time to think without interacting with me, that’s fine—I want to reduce the extreme “performance art” dimension of the live coding interview.
Comfort with their programming language of choice, showing that they really code professionally and have developed ease with their tools.
A methodical approach, as opposed to chaotic/guess and check; developing a mental model of the code that they can describe and walk you through. As an interviewer, I hope that they will bring a similarly scientific approach to their work.
Comfort with basic data structures; this is, to a degree, more of a signal between engineers that it is a direct evaluation of skills needed for success, but data structures do come up.
Write a program that can check whether a Sudoku solution is correct.
Write a program to find a solution to the N-queens problem.
Architecture
Why: Engineers have to design systems, which requires reasoning about technical trade-offs and seeing a picture bigger than an individual line of code.
Discussions of existing technologies, their properties, and trade-offs between them.
Design problems requiring a candidate to design a real system (typically on a whiteboard).
Exploring past projects, examining design trade-offs and alternatives.
Ability to maintain a reasonable level of abstraction, that is, avoiding getting sidetracked by small details and instead composing an overall picture of the system they are designing. Excessive focus on details is generally an obstacle to reasoning about system design.
Ability to articulate key design concerns and discuss trade-offs in their approach; a candidate should clarify what exactly the requirements of a system are and demonstrate a sound reasoning process that would generalize to a variety of problems. That means that they don’t just make good choices but that they’re able to articulate why they made them for this specific problem and why alternative approaches were rejected.
Staying on topic and solving the problem you’re asking about, not some other problem they want to solve.
Some knowledge of the key domain technologies (depending on their level of experience and the specificity of the role); particularly, more experienced candidates should usually show that they have made an investment in learning in their chosen domain, such that if asked to design a system, they could reason intelligently about the components available to choose from. I personally believe that this knowledge is quite undervalued in most hiring processes—a lot of work requires people to know things. Obviously, there may be exceptions to this criteria—junior candidates, or those transferring across domains, may show promise that justifies forgoing deep domain expertise.
Design Twitter.
Design an iOS app for online chess.
Tell me about a tough design problem you’ve worked on in the past. [Then, for important elements of the design] What were the alternatives, and why did you choose this approach?
Communication
Why: Most substantial engineering work requires collaboration.
How: I don’t suggest a dedicated interview for this area. Instead, this dimension should be evaluated in all interviews.
What to look for: You’re looking for two things: good listening and clear self-expression. Good listening means that the candidate is able to gather the essence of a problem from you, asking good clarifying questions as needed. Clear self-expression means that they can convey their own ideas back to you. There are no simple rules for evaluating this, but I suggest paying particular attention to whether a candidate has understood what exactly you’ve asked them and whether, when asked clarifying questions, they can answer on topic, completely, and succinctly.
Domain Knowledge
Why: In my opinion, performance on puzzly interview coding questions is overrated, and deep domain knowledge is wildly underrated. I am almost wholly unsympathetic to the oft-quoted, “I’d rather have someone who can learn than someone who has just memorized stuff”—personal experience is that when things get tricky, the people who deeply understand the systems involved, be they databases, languages, cloud systems, or whatever, will save your bacon, and people who “know how to learn,” admirable though they may be, will still be tying their shoes when the knowledgeable people cross the finish line. Try to hire engineers who know stuff!
How: An architecture interview can be a good place to dive in on domain knowledge; ask probing follow-up questions and see how deep a candidate can go on a given technology. I also once had an interview with a firm that just asked me 50 or so pure knowledge questions, one after another—I thought it was a radical and interesting approach given the norms of that time, and it might serve you well too.
What to look for: Knowing stuff, being able to contextualize and apply it in a fluid way as you ask questions (ref. organized thinking, communication).
Can you tell me what people mean when they talk about trade-offs between consistency and availability in database systems?
Can you tell me what happens when I type a domain name into a browser address bar and press “Enter”?
Organized Thinking
Why: Substantial engineering problems require navigating between high and low levels of abstraction (design and implementation), prioritizing between varied tasks, and identifying dependencies between tasks. You want a signal that your candidate will approach complex, varied sets of tasks deliberately, without becoming paralyzed by the variety of choices at any given moment.
How: This area should be emphasized in the architecture interview, which requires the most navigation of different levels of abstraction. However, like communication, it can be observed in every interview.
What to look for: You should look for a smooth flow from understanding requirements to identifying key design problems to choosing components. You should consider it a red flag if a candidate jumps rapidly between different levels of abstraction without trying to deduce stage N from stage N-1.
Attitude
Why: Two reasons. First, I haven’t yet met a level of brilliance that I consider to make up for disrespect—my experience is that one jerk can poison a whole team. Second, even the most perfectly prepared genius needs passion, dedication, and humility to adapt to changing circumstances, find common ground with others, and take responsibility.
How: We mostly take aim at attitude with questions about experience, interests, and goals, but it can surface through any question. An experience or “manager” interview is the obvious time to explore these areas, but I’ll often sprinkle a question or two at the beginning of a technical interview just to round out the picture.
What to look for: I look for two things: passion (enthusiasm for technology and/or problem-solving) and a collaborative attitude (respect for others and willingness to compromise).
Passion doesn’t mean effusion—quiet people can do great work and never singing songs about it. However, I expect engineers to articulate some interest in technology or in real-world problems—problems they’d like to tackle, skills they’d like to learn, technologies they think are cool.
It’s easier to find red flags when it comes to interpersonal attitude than it is to prove excellence. Candidates should know better than to bad-mouth past colleagues or show contempt for your questions.
What are you looking for in the next few years of your career?
What technologies are you interested in working with?
Tell me about a time you had to deal with conflict on a project.
Tell me about a coworker you’ve admired in the past and what you learned from them.
The Nuts and Bolts of Interviewing
This section will describe how I personally plan and execute an interview; it’s not the only way, but it’s a fine way to start.
First, before the interview, I review a candidate’s resume, decide what questions I’ll ask, and plan the schedule down to the minute. Why such careful planning? This is because the most common failure mode for interviewers is to fall down a rabbit hole on a minor point and end up with no signal, and that is 100% on the interviewer when it happens; I plan my time and watch the clock.
I start every interview by smiling; introducing myself by name, team, and specialty; then telling the candidate what we’ll cover in the interview. In a competitive hiring environment and as a matter of principle, the candidate should feel at all times that they’re welcome, that your energy is positive, and that their experience is well-organized; a smile and an overview are a good start. You’ll cultivate your own manner, but I try to be professional, calm, and breezy—commiserating and laughing when I can, trying to lower the intensity and praise progress. Some interviewers favor a cold, withdrawn style, thinking that it forces the candidates to “step up” more—I disagree, as comfort with jerks isn’t an important criterion for me in a candidate.
I almost always follow with a gentle warm-up question about a simple technical problem. This gives people a chance to loosen up for the tougher questions and also lets me see how they communicate material they know quite well. For example, I might ask, “tell me about what a hash table is and what its properties are.”
If I have an hour, I might also ask a question about their experience; either way, I’ll allocate most of the interview for the main question, either a coding or design problem. I suggest you source some of these from colleagues or from the many Internet problem banks.
Finally, I’ll allocate five minutes for them to ask questions, which is also a chance for me to chat with them, encourage them to relax, and sell them on why my company is a great place to work; even if they failed my interview, they may ace every other one, and I want to do my part to close them.
45-minute interview
2 minutes: Introduction
3 minutes: Interests question
5 minutes: Basic data structures question
30 minutes: Coding problem
5 minutes: Q&A
During the interview, I proactively manage the clock; you should feel free to redirect the candidate if they get stuck or just to say, “Great! That’s an interesting discussion, but we need to move on to the next question to stay on schedule.” My schedule bakes in some time for the candidate to struggle, and when they do, I let them (a bit); you want to see how they react when really challenged. However, I don’t let a candidate make zero progress for more than a few minutes; at that point they’re probably really stuck, and there’s no more to be gained from withholding a nudge.
Finally, throughout the interview, I take detailed notes; I use my own shorthand to capture what I say, what they say, and general observations (something like italics for what I say, quotes for what they say, brackets for observations). This is quite important, because you will forget most details within the first hour after the interview. Some people prefer paper to a laptop for this purpose, because a laptop can really feel “between” the interviewer and the candidate.
Decisions
You’ve heard my skepticism about any supposed objectivity of hiring decisions. Therefore, the only way to make decisions is intuition, and the main way to calibrate is experience. So, your first few decisions are going to be very rocky, and that’s okay. Ask your questions to a bunch of candidates, and you’ll get a sense for where people struggle and what average performance looks like; see how your respected colleagues evaluate candidates you’ve interviewed, and try to refine your process. I’ve had a great experience with mock interviews—interviewing a great colleague as if they were a candidate. A great teammate should crush the interview—if they do, you know your question at least isn’t unreasonably hard, and they may have good feedback on what is and isn’t clear.
Software Methodologies and the Generality of Good Engineering
All teams have systems for allocating their resources and planning their execution; those methodologies, which fill countless books, can range from the highly formalized (heard of a Certified Scrum Master?) to the totally improvised. This section will prepare you for some of the common methods you may encounter in the wild.
First, the big picture. In my (possibly controversial) opinion, engineering fundamentals are universal across methodologies. Your priority is always to excel at project management, collaboration, and writing working software, and you can do those things well in any framework much the same way: project management is always about understanding dependencies, staying organized, and conservative estimation; collaboration is always about crisp communication, empathy, and efficient use of your colleagues’ time; software that works is software that works. So, don’t overthink it!
Still, it’s worthwhile to understand your team’s system, and we can extract useful insight from the formal methods; we’ll discuss a few common elements. Note that if your team uses any formal method, you should study it on your own to see where they’re coming from.
Yearly, half-yearly, or quarterly planning are extremely common features of industry life. Companies set a regular cadence where engineering teams and stakeholders—product managers, businesspeople, designers, etc., etc.—meet and/or fight each other to the death to set goals. You should remember two things about these processes. First, the pressure will always be to commit to more, never less; try to gently hold your ground when you believe goals are too ambitious. My refrain is usually along the lines of, “I can only be frank about what I think is realistic.” Second, you and your manager will be held accountable for your team’s goals. That means that once your team has committed, you should care rather a lot about delivering.
Day to day, it’s quite common to see teams do ad hoc planning, that is, distributing work to individuals’ queues as it comes in or distributing individuals to work as they finish tasks. This can co-occur with fixed planning cycles, which may decide only high-level goals. Some people find ad hoc planning frustrating and chaotic; I happen to feel that improvisation can be fun and powerful when a team is well-aligned. Hold on tight, and accept that you don’t always need to know what you’ll do a month in advance! Ad hoc processes demand one specific reminder: when you finish something, you absolutely must go find something else to do, asking your boss if necessary. We don’t sit still just because no one is pushing us!
Agile Software Development
Agility is an organization’s capacity to adapt to changing circumstances: swift execution, lightweight planning processes, and openness to course corrections. “Agile Software Development,” or “Agile,” is a class of formal development processes intended to encourage that quality.
Agile principles argue for short development cycles that integrate requirements gathering, design, and development, constantly shipping small increments of working software, bringing together stakeholders and developers to review and assess, and replanning as circumstances change. You can reference the Manifesto for Agile Software Development for a sense of the genesis of this movement, but Wikipedia, or a book like Essential Scrum, will offer much more detail.
My experience agrees with these principles: I’ve seldom seen complexity foreseen in detail, and a short-duration, iterative, open process can contribute to (but don’t guarantee) a real ability to swivel toward newly discovered problems and opportunities. Scrum and Kanban are probably the best-known attempts to systematize these concepts. Scrum, the more common, is described below, but it’s a good investment to learn something about Kanban’s powerful visualizations on your own.
Scrum
Scrum organizes the software development process into a rhythm of self-contained “sprints,” typically periods of 1–4 weeks, and codifies specific roles and meetings to structure those sprints. Its essential principle is collaboration, with all team members contributing to planning, implementation, and team introspection and daily helping each other overcome blockers. I find that many teams borrow at least some Scrum methods, especially 2-week sprints as a planning cadence.
First, a Sprint Planning Meeting where the team works together to set tractable goals for the sprint.
For the duration of the sprint, a short daily “scrum” or “stand-up,” where team members discuss their progress, help each other resolve “blockers” (problems impeding their progress) and cross-pollinate useful information.
At the end of the sprint, a Sprint Review and a Retrospective for reviewing (and celebrating) progress, demoing results, and identifying opportunities to improve.
Asynchronously, the team regularly reviews and prioritizes (“grooms”) a “backlog” of future work. You’ll find that teams vary in their adherence to the orthodoxy; sometimes “Scrum” means a radical commitment to iteration and collaboration, and sometimes it just means having a daily meeting.
My experience is that formal scrum-like processes, where a team insists on sprint-length deliverables, can work beautifully for simpler, cohesive projects, where complexity is foreseeable, work easily decomposes into value-adding pieces, and team members share a backlog. I’ve seen them turn quite awkward and meeting-heavy on the deepest projects, where engineers regularly uncover unpredictable complexity and daily facetime may not add value.
I’d add a few specific tips on top of the general purpose advice above. First, Scrum demands hyperefficiency in meetings, because you meet every day—prepare concise, informative updates in advance. Second, try to make the most of Scrum’s daily facetime with teammates: listen, make the most of what teammates share, and try to unblock them. Third, a tight planning cycle can be relatively intolerant of detours (i.e., in some cases less agile); you may need extra diligence about avoiding even value-adding detours.