Chapter 27. Documentation in Scrum Projects

We’ve all heard the common myth, Agile means no documentation. While other agile fallacies exist, this is a big one, and it could not be farther from the truth. Good agile teams are disciplined about their documentation but are also deliberate about how much they do and when. In this chapter’s story, we find a duo struggling to explain that while they won’t be fully documenting everything up front, they will actually be more fully documenting the entire project from beginning to end.

Carter and Noel looked at each other, then back at Ashley, their manager, before replying. They had known this conversation was coming but didn’t realize they’d be accosted in the hallway by an obviously harried Ashley when it did.

“Listen, we can document everything up front like you ask,” Noel began, as she and Carter moved to stand close to Ashley’s doorway. “But we don’t think it’s the best approach. Things change, and we cannot promise you that things will go as planned. Further—” Ashley stopped typing and looked up, interrupting Noel mid-stream.

“Look, I don’t want to argue about something as basic as documentation. I just need it on my desk by Friday.”

Carter spoke up.

“Ashley,” he began. “Can I have five minutes to communicate a different approach? I know you have a full plate, but I think it’s important for you to try to understand this point before we table our discussion.”

Ashley glanced at her watch and nodded. “Five minutes. Go.”

“When I was in college, I worked for our university newspaper,” Carter explained. “I was a sports photographer and always attended local football games with the sports writers. I was on the field, and they were in the stands.

“It probably won’t surprise you to hear that not one of those sports writers came to the football game with the story already written. Now, they might have done some research on the players. They might have talked to the coaches about their game plans. They might even have asked me to be sure to get some shots of particular players. But they never wrote the article before the game even began.

“That’s kind of what you are asking us to do with the software. You want the complete story of how this application will unfold, including the final game score, before we’ve even started playing,” said Carter.

Ashley replied, “Well, that’s how we get things done around here. Without the up-front documentation—design docs, use cases, master requirements docs, architectural designs, and release plans—I won’t get project approval, and I can’t be sure that you two and the rest of the team understand what we need to build.”

“Right. I get that,” agreed Carter. “It’s not unreasonable for you to want some information before we get started. And you should expect to receive frequent updates from us on what’s going on with the project. After all, the reporters I used to work with would take notes and write snippets of the article about the game as it was unfolding. Periodically, we would get together to discuss the angle they were working on and some of the shots I had captured so far.

“But to ask us to tell you what the software will look like, exactly how much it will cost, and precisely when we’ll be done is like asking us to predict the final score of the football game. We can tell you how we think it’s going to go, but when things are changing and unfolding, it’s difficult to predict all the details.”

Ashley nodded. “But things aren’t always that volatile with our projects. We know basically what we want this to look like. It’s only some of the features we aren’t sure of.”

“Right,” chimed in Noel. “And on projects where we can nail down most of the variables and have a clear picture of the final product, we can give you more up-front documentation.”

Carter nodded. “To go back to my sports writer analogy, there were times when one team was clearly dominating—the game was a blowout. In those cases, the reporters had their stories mostly written by halftime. They’d already come up with the headline, filled in a lot of the details, and were just waiting until the end of the game to add the final stats and score.

“Most times, though, the games were close and the outcome uncertain. In those cases, the reporters would keep filling in the skeleton of the story with the events as they happened in real time. They would come down to the field at halftime, and we would discuss the unfolding story. We’d strategize and say, ‘If the game goes this way, we’ll take this approach. But if it goes that way, we’ll take this other approach.’

“Likewise, the level of detail in our documentation at any given point in the project should depend on how certain we are that things aren’t going to change.”

Ashley leaned back in her chair with her hand on her chin, deep in thought. Noel decided to go in for the kill.

“Ashley, remember the 1989 San Francisco earthquake, or the quake and tsunami in Japan? Or when either Reagan or Kennedy were shot?”

Ashley nodded.

“Well, you would notice a trend in all these events. In the initial accounts, the media headlines conveyed the tragic event, but with very few details. All they could tell us at first was generally what had happened (explosion/quake/tsunami/shots fired), when, and where. Why? Because the events were still unfolding, and that was all anyone knew. As the reporters on scene learned more, they added the new facts and changed the headlines.

“All the little updates, facts, and details were important to capture in real time, even if they later had to be changed to reflect new information. Without continuous briefings, many of the details surrounding the events would have been forgotten in the chaos of the devastation. The reporters didn’t try to write more up front than the facts they knew. Instead, the reporters recorded what they did know as they went along. Later, after the details had solidified, they went back through the various articles and wrote a larger, encompassing synopsis that outlined the specific event from the initial reports to the current state,” Noel explained.

“That’s what we’re suggesting we do: Make our documentation a story in progress. Are we making sense?” asked Carter.

Ashley sat forward.

“I think I get it now. What I originally heard you say was, ‘I can’t give you documentation.’ But what you’re actually saying is that you will document certain things up front, most things in real time, updating them as necessary to reflect reality, and some things after the fact. But what does that mean in terms of software exactly? I need certain documents to get the project approved, such as the master requirements documentation outlining all the use cases.”

Noel answered, “We will provide that MRD, but it will be in the form of a product backlog, with stories at different levels of detail. Some of the stories will be headlines only. Others will be fully fleshed out, maybe to the point of use cases. For example, the team has a good idea of the architectural direction, which gives us insight into what stories to build for the first six or so sprints, but after that, things get fuzzy. We know that you want us to capture the details, and we have those details for the higher-priority stories, but we don’t have them for the stories off in the distance. For those, we might only have headline-level information.”

Ashley asked, “But if we don’t capture details up front, won’t they get lost? And don’t you then have to spend a significant amount of time getting and documenting that information during the project, if they can be found at all? How can you ensure that what is being built is correct?”

“Remember, like any good journalist, we will document as we go, as soon as we can, without doing too much. That way when we get to the point where the UI stabilizes, let’s say, we can create the more detailed documentation that the company needs and that we need to deliver in the form of user manuals and such. We won’t lose our details because we’ll be going over those details every sprint with the team, documenting them as part of our definition of done, and checking with the stakeholders to ensure we’re on track, to ensure that what is being asked is what we are building and matches what the customers meant. Keeping documentation up to date a little at a time is just part of the cost of building working software. If things change along the way, we will update what we have written to reflect the change. It’s a balance between stability and volatility. The more volatile something is, the more careful we need to be in what level we document. If it’s stable, we can do something such as a large database diagram model in a tool. If it’s volatile, we might just draw a picture on the whiteboard—again, both are documents, database models to be exact, but they are very different in terms of formality,” finished Noel.

“So, are we on the same page?” asked Carter.

“Yes,” said Ashley. “I get it now. I think this is a good approach and something that I will advocate, provided you give me regular feedback so I can update executive management. But I still need the big headlines by Friday. Agreed?”

“Agreed,” said Carter and Noel together.

And that was that.

For example, imagine you are in a university world history class. You get to the point in the class when it’s time to discuss Western European history. Your professor says, “I want each of you to buy my new book Western European History: The 30th Century. Come prepared for an exam on the first five chapters in two weeks.”

You would probably look around the room, wondering if what you just heard was correct and ask a fellow student, “Did he just say thirtieth century history?”

Common sense tells you that without time machines, it is impossible to read a factual account of future events—they haven’t happened yet! Sure, there are predictors and indicators that suggest what might happen, but nothing is certain. This then begs the question: If this approach is wrong for a university class that may reach a few thousand, why is the exact same approach accepted when developing software that has the potential to reach millions?

Before we’ve begun any work on a project, we are often asked for exact details as to what will be delivered, by when, and at what cost. To determine these things, teams often write volumes of documents detailing how the system will work, the interfaces, the database table structures—everything. They are, in essence, writing a history of things that have yet to occur. And it’s just as ludicrous for a software team to do it as it would be for your history professor.

That doesn’t mean we should abandon documents, and it doesn’t mean that we should leave everything until the end, either. A certain amount of documentation is essential at each stage of a project. Up front, we use specifications or user stories to capture ideas and concepts on paper so that we can communicate project goals and strategies. When we sign off on these plans, we agree that what we have documented is the right thing to do.

The question, then, is not whether we should document, but what we should document and when. The answer has everything to do with necessity, volatility, and cost.

Over time, the only representation of the original knowledge becomes the code itself, which by now is something we can run but not exactly understand. It has become a process, something we can operate but no longer rethink deeply. Even if you have the source code in front of you, there are limits to what a human reader can absorb from thousands of lines of text designed primarily to function, not to convey meaning. When knowledge passes into code, it changes state; like water turned to ice, it becomes a new thing, with new properties. We use it; but in a human sense we no longer know it.

Why is Ullman’s concept of code as a process that changes state important? Because we need to realize that, in a human sense, we use the system and we know the system. That is why we document.

So, what is essential to document, and what is needless work? Much of that depends on the type of system you are building and the way in which you work. Teams that are colocated need to document less than do teams distributed across continents and time zones. Teams that are building banking systems need to satisfy more regulatory requirements than do teams building marketing websites. The key is to document as much as you need and nothing more.

End-user manual

Operations user guide

Troubleshooting guide

Release and update manual

Rollback/failover manual

User stories and details

Unit tests

Network architecture diagram

Database architecture diagram

System architecture diagram

Acceptance test cases

Development API manual

Threat models

UML (Unified Modeling Language) diagrams

Sequence diagrams

We didn’t write all these documents before the project began. And we didn’t wait until the final sprint to start them either. We did them as the information became available. Many of the user stories, for instance, were written up front. But some of them were changed, and others were added as the project progressed and requirements became clearer. Our unit tests were written as we coded. And at the end of every sprint, we updated the end-user manual to reflect new functionality. We included in our definition of done what we would document and when we would write it (see Chapter 7, “How Do You Know You’re Done?”).

Let’s look at a few approaches we can take when documenting our projects:

Document heavily in the beginning.

Document heavily in the end.

Document as we go along.

The benefit of this approach is that people feel more secure about the system being built. The major drawback is that this sense of security is misleading. In point of fact, though a great deal of time, effort, and money has gone into writing the documents, no working software has yet been created. The chances of getting everything right up front are marginal on stable projects and next to zero on volatile projects. That means factoring in costly rework and extra time. Chances are good that these high-priced, feel-good documents will turn into dusty artifacts on the project bookcase.

The benefits of this approach are that working software is created quickly and that what is eventually written should reflect what the system does.

There are many problems with this approach. People often forget what was done and when, and what decisions were made and why. Team members on the project at the end are not necessarily the people on the project in the beginning; departing team members take much of their knowledge with them when they go. After the code for a project is complete, there is almost always another high-priority project that needs attention. What usually happens is that most of the team members move on to the new project, leaving the remaining team members to create the documentation for the system by themselves. Countless hours are spent hunting for data and trying to track down and access old team members, who are busy with new work and no longer have time for something “as insignificant as documentation.”

Though saving documentation until the end is cheaper in the beginning because more time is spent on actual software development, it is usually expensive in the end because it can hold up a release or cause support and maintenance issues, as it will likely contain gaps and faulty information.

The product owner works with the stakeholders and customers to build the requirements while the team works with the product owner to achieve emergent design and architecture. The team keeps the code clean, creates automated tests, and uses code comments and other tools to slowly build other required documentation for the system, such as the user manuals, operations guide, and more.

The one drawback is that it does take a little longer to code when you document as you go than it would to fly through the code without having to write a comment or update an architectural diagram. This drawback is more than offset, though, by the benefits. There is less waste, less risk of eleventh-hour holdups, and more emphasis on working software. Much of the documentation is updated automatically as changes are made to the code, reducing maintenance and rework costs. Just as news reports capture the details of a story for posterity, real-time documentation of decisions and behavior minimizes gaps in knowledge and creates a living history of the software for future teams and projects.

Let’s say, for example, that I’m writing the user manual for an update to Adobe Lightroom (my current favorite piece of software). I’m in sprint planning, and the product owner explains that the story with the highest priority is “As an Adobe Lightroom user, I can export a series of photographs to Adobe Photoshop so I can stitch them together to make a panorama.” As we’re talking through that story, I recommend that we add “update user manual to reflect new functionality” as one of the acceptance criteria for that story.

As I write the code or as I finish the feature, I would also edit a document that provides the user instructions on how to use the feature. Depending on how stable the feature is, I might even include screenshots that walk the user through the instructions for both Lightroom and Photoshop. If the feature is less stable, meaning the core components are built but the user interface team is still hashing out the user interface through focus groups, I would document the behavior but probably only include placeholders for the screenshots. The key here is that the story would not be done until the user manual is updated.

Updating the user manual would be appropriate to do at the story level, as I described, but could also be accomplished at the sprint level. For instance, if we have several stories that revolve around user-facing functionality, we might add a story during sprint planning that says, “As a user, I want to be able to learn about all the new functionality added during this sprint in my user manual.”

What I am doing is balancing stability versus volatility of the feature to determine how deep I go and when. It would not, for example, be prudent to make updating the user manual part of the definition of done for a task. Too much might change before the story is complete. Nor would it be acceptable to wait to update the user manual until right before a release. That’s far too late to start capturing the details of the new behaviors.

When determining when to document your own systems, you also should balance cost, volatility, and risk. (For more on determining your definition of done, refer to Chapter 7.)

Eschewing extensive documentation up front does not mean you are off the hook for a project signoff piece. But it does mean that the piece will look different to your stakeholders than it has on other projects. Rather than give them the specific artifacts they request, answer the questions they are asking in regard to schedules and requirements in the most lightweight way possible for your project and situation. A PMO might, for instance, ask for a Microsoft Project plan, but what the PMO really wants to know is what will be done by about when. By the same token, a stakeholder might ask you for a detailed specification, when what she really wants to know is, “Are you and I on the same page with regard to what I’m asking you to do?”

Signoff and approval will occur early and often. The product owner will hold many story workshops to build the product backlog, work with the team to build the release plan, and then communicate that information to all interested parties, soliciting enough feedback to ensure that the team will deliver what the stakeholders had in mind (which is rarely exactly what they asked for). The documents the product owner uses for these tasks are only a mode of transportation for ideas and concepts, and a document is not the only way to transfer those ideas. Up-front documentation can just as easily take the form of pictures of whiteboard drawings, sketches, mockups, and the like—it does not need to be a large formal document.

The beginning of the project is when you know the least about what you are building and when you have the most volatility. What your stakeholders need is the peace of mind that comes from knowing you understand what they need and can give them some idea of how long it will take to deliver. Expend the least amount of effort possible while still giving them accurate information and reassurance. At this point in the project, everything can and will change.

Decide—Determine what you need to document for your project and when it makes the most sense to produce that documentation. Some things, such as code comments, are easy to time. Other items, such as threat models, are more difficult. Work as a team with your product owner to determine the must-have documents at each stage of your project.

Commit—Once you have a documentation plan, stick to it. Put it in your definition of done. Hold yourselves accountable. Documentation is never fun, even when it’s broken into small chunks. Remind your team that a little bit of pain will eliminate a great deal of risk come release time.

Communicate—If this is the first project to move forward without extensive up-front documentation, the stakeholders will be nervous. Help them out, especially at the beginning of the project, by sending frequent updates, pictures of whiteboards, and any other documents that are produced. Do as your math teacher always told you: show your work. Seeing working software and physical artifacts goes a long way toward calming the fears of even the most anxious executives.

Invest in automation—Documentation is easier and ultimately cheaper if you invest a little time in automating either the system or the documentation itself. For example, if you can create an automated script to compile all the code comments and parse them into documentation, you’ve saved a manual step and instantly made your documentation more in sync with the actual code. It’s also much easier to document acceptance test results and API documents automatically than it is to do it manually. On the flip side, you might find that automating the features themselves can save you a lot of documentation work. For example, a manual installation process might require a 40-page installation guide; an automated installation process, on the other hand, probably needs only a one-page guide and is better for the end user as well. Whenever possible, automate either your documentation or the features it supports. The results are well worth the investment.

Being agile does not equate to no documentation; it means doing timely, accurate, responsible documentation. Make sure that documentation is equally represented in your team’s definition of done alongside things like code and automation. Remember that when change happens, it’s not just the code that changes—the entire software package that you are delivering changes, documentation included. Lastly, remember that as much as you might wish otherwise, documentation is a part of every software project. When you do a little at a time and automate as much as possible, you’ll find that while it’s still an obligation, it’s not nearly as much of a chore.

[ULLMAN] Ullman, Ellen. Salon.com. http://www.salon.com/technology/-feature/1998/05/13/feature (accessed 18 November 2010).