Setting Traps for Information
In addition to searching for new information and trying to judge its significance, you can also set traps to capture significant information that bears examination.
Danger Bearings
You can use estimates and measures based on estimates as guidelines to warn you when you’re tending toward disaster. You want to plot your progress in a way that assures you’re in a safe zone, or warns you if you’re getting close to being outside.
When navigating a boat, such a guideline is called a danger bearing. You plot a line on the chart from some visible point to some hidden danger. As long as you stay outside of that line, you’re OK. You may think that by knowing your starting point and knowing the direction you’re traveling, you would also know you’re on a safe course. It’s not that simple, though. As in software development projects, unseen currents can carry you off that course, even though you’re still headed in the direction you planned. To make sure you’re not drifting into danger, you can check whether you’re outside of that line by checking the bearing to that visible point. Some permanent navigational marks have designs to keep you on the safe side of some hazard. The danger bearing is built into a lighthouse by a red filter in front of the light. If you see red light, you’ve crossed the line and are heading into danger.
One way to notice obsolete estimates is to record guard conditions—perhaps something similar to “if we haven’t achieved this outcome by May, then this estimate is suspect.” That lets you know that the estimate is obsolete. You can revisit it, and the larger estimates that include it, and decide what to do.
More generally, stay out of danger by using a BurnUp Chart. (See BurnUp Charts.) Estimate the minimal amount that will be a valuable release, for whomever you need to please. Since you’re looking for safety, make sure your estimate errs on the high side of what that minimum valuable amount might be. Express that in measurable terms, so you can tell when you reach it. Split it into smaller, but still measurable, increments of progress. As things move forward, look to see if the current trend line meets this bare minimum success criteria before a hard deadline, preferably well before. Of course, you’d rather deliver more than the bare minimum. But concentrate on reaching that level of success first, and then add enhancements as long as there is time to do so.
Hypothesis
It’s helpful to position your estimates as hypotheses rather than predictions. You can test your prior hypotheses against what you actually achieve. This allows you to measure how far off your assumptions are, and to notice things you didn’t know.
Experimentation is a powerful learning tool, and the scientific method rests on the performance of experiments to confirm or deny a proposed hypothesis. When I was young, I performed “scientific experiments” by mixing chemicals together to see what they would do. I learned that most random concoctions from my chemistry set would make a brown liquid that was often hard to clean out of a test tube and that sometimes they would create very smelly brown liquids. These were not really experiments, however. These were activities, and because I had no hypothesis to prove, I didn’t really learn anything useful. Unless you can propose a hypothesis in advance, you cannot design an experiment to test it. Until you test the hypothesis, you haven’t really learned anything.
“In general, we look for a new law by the following process: First we guess it; then we compute the consequences of the guess to see what would be implied if this law that we guessed is right; then we compare the result of the computation to nature, with experiment or experience, compare it directly with observation, to see if it works. If it disagrees with experiment, it is wrong. In that simple statement is the key to science. It does not make any difference how beautiful your guess is, it does not make any difference how smart you are, who made the guess, or what his name is – if it disagrees with experiment, it is wrong.” – Richard Feynman, The Character of Physical Law [Fey67]
When you estimate how long it will take, or how much it will cost, to implement a desired amount of software functionality, you create a hypothesis that you can test. Your hypothesis may not be of enduring and universal value as a hypothesis that predicts physical laws, but it may still be extremely valuable to you in your situation. You can test your hypothesis, with all of its explicit and tacit assumptions, against the hard truth of reality. When your hypothesis is found wanting, you can consider which of its supporting assumptions is incorrect, or if there are necessary assumptions which are missing.
So, set up a few intermediate points that are on the way to your first milestone. Estimate these. That’s your first hypotheses.
How early can you test the correlation between the estimates in your plan and the actual rate of progress? When work is performed in large blocks and integrated at the end, it’s far too late for effective replanning when the difference is noticed. How can you validate your plan earlier?
I recommend setting your first hypothesis as within a couple weeks to a couple months, preferably less than 10% of the way to your first milestone of importance outside the project. While you need to inform others when the situation changes (see Evaluating and Changing Plans), you first want to inform yourself. Perhaps you can correct the trajectory of the project before it becomes imperative to report it up the chain. Failing that, you’ll have more time to develop a realistic plan to suggest.
Working in small increments and continuously integrating can help, but you also need to verify those small increments are actually done. Using estimates of “the database layer is 90% done” won’t help, as you’re back to depending on unverifiable estimates. Rather than creating small increments based on the building blocks of your project, try using small increments of functionality. These functional slices will touch numerous building blocks, but can be tested unambiguously as to whether or not they work as desired. Even better is if they can be put into production use and tested by real work. This roots
out the unexpected undoneness that can throw your projections off so much. See Reliably Measuring Functionality for more discussion on measuring progress by testing the functionality of integrated code.
That’s a lot of prescriptive description about how to start your project using estimates as hypotheses. Let’s look at how this might play out in concrete circumstances. As you read this story, imagine yourself in the position of Jesse, and think about how you would normally handle the situation. Also think about how it would feel to handle it the way Jesse is handling it.
Planning a New Project at Riffle & Sort
World Of Friends has been a payroll customer of Riffle & Sort since they opened their doors. It is a small but busy nonprofit that would rather focus on their mission of helping arrange student and cultural exchange programs to further world understanding and peace, than deal with the humdrum bookkeeping.
They’re finding that the world is getting more complicated, and there is more and more humdrum paperwork involved in making a successful exchange program work. Rather than increasing staff to handle the workload, they’ve come to Riffle & Sort for help.
While the project may be about automating the current paperwork processing, the goal is a higher-level ambition, “furthering world understanding and peace.” A computer system cannot do that, but it can help people do that.
Eliciting the Requirements
Jesse, the project manager and onsite proxy for the customer, had interviewed several people at World Of Friends to get an idea of what they needed. The world had indeed made things more difficult for cultural exchanges. The process was something like this:
-
Students wanting to participate would fill out an application form, writing several essays about who they were and why they wanted to participate.
-
This form would be reviewed by World Of Friends to select those most likely to do well in the cultural exchange program. Since the students would be hosted in private houses, it was important to choose people who would leave a good impression with the hosts.
-
Those students selected as applicants by World Of Friends would be sent a letter outlining next steps. Accompanying the letter would be various government documents that would need to be filed to get permission to come and live for an extended time in another country. The visas, in particular, would need official government background checks. These would require fingerprints. Since the rules were complicated and frequently changing, these documents were to be sent back to World Of Friends for submission to the appropriate authorities.
-
In parallel to the official background checks and visa processing, World Of Friends would conduct their own background check in the applicant’s home country. This was intended to turn up any red flags for behavior that would affect the host family.
-
Only after both of these investigative paths had been completed, a World Of Friends adjudicator would then make a final decision on the applicant. Part of this procedure was making sure that all the proper documentation had been supplied, all questions had been satisfactorily answered, and necessary government permissions had been given.
-
Once the final acceptance was made, World Of Friends would match up each applicant with a prospective host family based on the stated preferences from both. A letter of introduction would be sent to the prospective host family, describing the applicant and providing the essays they had written on the original application. The host family might refuse for any reason. They might have seen something in the description or essays that gave them a bad feeling. Or they might have had a change of heart about being a host family for completely unrelated reasons. It really doesn’t matter what the reason; another prospective host would be chosen and the same letter of introduction sent to them. This could happen multiple times until either a host family accepted or World Of Friends ran out of prospective hosts in the designated location.
-
Finally, a letter of acceptance introducing the host family would be sent back to the applicant. This letter would contain information on dates to arrive and how to make travel arrangements.
These are Jesse’s notes on World Of Friends current work process. It’s true that the people Jesse interviewed asked for these activities to be automated as much as possible. With a project manager less savvy than Jesse, Riffle & Sort might have been tempted to try to fulfill this request as stated. Jesse knew, however, that what people ask may not accurately reflect what they desire. Also, not everything that people mention has equal importance. Keeping the goal in mind, Jesse started analyzing the request with a view toward simplicity, feasibility, risk, potential value, and opportunity.
Assessing the Constraints
Jesse thought about the best ways to ensure that World Of Friends was happy with the result and stay within their budget. While they wanted to automate the process as much as possible, they realized they wanted to keep some human control at appropriate points. In any event, they needed to have the system ready to go in early January for the coming year’s cycle of applications. The deadline for applications was mid-March, so things would be in high-volume mode by then.
It was currently early April, and World Of Friends was recovering from the big bulk of work handling this year’s applications. But there was plenty more work to be done, and it would pick up again when the background checks started coming in from the government agencies and the applicants’ home countries. Right now would be a great time to work closely with World Of Friends, but the development team was putting the finishing touches on another project. They wouldn’t be able to turn their attention to this one until the beginning of May. That gave them eight months, more or less, to have a working version. It had to be especially ready for the entry and initial triage of applications. There would be time to enhance later operations, such as the matching with host families.
Most projects need to fit into some sized box defined by time and money. All projects need to fit the constraints of feasibility within the capabilities that can be brought to bear on them. If you cannot satisfy these requirements, then people will be unhappy no matter what you do well. This implies that a less-capable system that meets these rudimentary constraints will be better received than something fancier that doesn’t.
Analysis and Planning
What’s the simplest system that could work? Using the computer to be a filing system for scanned paper forms was definitely the simplest, Jesse thought. They could test this with forms from previous years, blacking out a bit of personal data to meet privacy standards. This would let the team concentrate on the workflow and decision points. Online entry of the original application was an obvious time-saver. This was the highest-volume processing to be done.
Submitting the government forms was the riskiest part of the operation. Jesse felt sure that the government would be the most stringent on what they would and would not accept. And it was likely there would be unexpected communication delays and incomplete descriptions of why a submission was not accepted. This should go early in the plan to mitigate the risk of unknowns outside Riffle & Sort’s control.
World Of Friends would be matching applicants with host families this summer; it would be good to work on that aspect while they were doing it, to capture the little details that people forget about after they get past them.
Jesse considers simplicity, risk, and variability between different deadlines in developing an initial plan. This plan is, so far, without a schedule.
A Tentative Schedule
Jesse started making lists of the inputs the system would have to accept, the outputs it would send, and the decision points. For each of these, someone would have to work out the details, but it was enough to list them for the moment. With that list, they could document the flow and check its accuracy with World Of Friends. They could also estimate each of these processing points independently, assuming they would build the basic flow first. That basic flow could fake the inputs, stub out the output, and turn the decision points into simple buttons. That should be possible in the first two weeks, Jesse thought. Then everything else can be bolted onto this rudimentary flow as it’s fleshed out. If that simplistic flow isn’t finished in four weeks, it’s a sign of major problems.
Not having any better data, Jesse counted the inputs, outputs, and decision points and divided the eight months by that. "Oh, better figure the government inputs and outputs as double the others," Jesse thought, and recalculated. "That’s about two weeks each. Sounds barely doable, but since we can push out some of the later details after a January initial-release deadline, we should be in good shape. And some outputs, like sending letters, should be easy after the first one is done. I’ll check if anyone in the development team can help me sanity check these estimates next week."
Notice how Jesse set an initial danger bearing of four weeks to get a rudimentary flow working. That includes the expected two weeks of development work, and any other “getting started” delays that might arise. In addition, Jesse already has in place a series of hypotheses that each input, output, or decision point can be done with two-weeks work (except for the interfaces with government systems). That’s a good setup for alerting if things are going astray. Some of the alternate flows, such as rematching with host families, are expected to be included in these estimates. Others that are less likely, such as an applicant having to drop out, may not be. If there’s time, the system can be enhanced to include them. If not, they will be low-volume enough to be handled with manual work-arounds.
