3.2 How Quickly do My Teams Learn From Users?
In a complex, unpredictable, and dynamic market, it’s crucial that teams know quickly whether they are doing the right work.
Introduction Questions
1. Lately, what do users give as feedback?
2. How much time is spent per week on learning from actual usage?
The learning loop provides insight into whether the teams are doing the right thing. But does this insight come quickly enough? Is feedback requested quickly enough? It is not possible that the teams will only know in a year’s time whether they are doing the right thing today. This can be compared to driving for half an hour blindfolded or, as a cook, not wanting to know whether guests like the food. In a complex, unpredictable, and dynamic market, it is crucial that teams know quickly whether they are doing the right work. They have to know whether what they make is still in line with the rapidly changing needs. It is therefore important that the learning loop is completed quickly—in other words, to have a high learning speed. The sooner and more quickly it becomes clear whether teams are doing the right thing, the better teams can respond to unexpected situations. But can this speed be measured? And if so, how?
Tool 5: T2L
There is a good way to determine how quickly the feedback from users is processed or how high the learning speed of a team is. For this we measure how long it takes to go through Steps 2 through 5 of the learning loop (i.e., from “Build” through “Learn”). This time is called the time to learn (T2L).
T2L |
Time to learn (T2L) is the total time needed from sketching an idea or improvement, building it, bringing it to the users, and learning from their usage. The T2L includes refining and designing, developing, testing, and integrating it, deploying it to market, letting users actually use it, gathering statistics and feedback on that usage, studying these results, and actually learning from it. |
This metric is based on the lead-time metric often used in manufacturing and the time-to-market metric. These metrics measure the internal process until just before the customer receives it. They don’t incorporate the feedback and data from actual users. In complex environments, this feedback is crucial to know whether the market has changed and how to continuously improve the actual customer impact. It’s the role of the agile leader to improve the environment of his teams so that they can learn faster.
The shorter the T2L, the greater the learning speed. This is comparable to traveling from Amsterdam to Paris. The shorter the travel time, the higher the speed must have been. So by shortening the T2L, the learning speed is increased. By improving the T2L, the teams learn faster from actual usage.
Figure 3.2 show some examples of calculating the T2L.
Figure 3.2 Calculating T2L
In the first example, the T2L is ten months. To make this concrete, if a management team makes the strategic choice to put a new product on the market in early February, they will know in mid-November whether they have made the right decision. Only then will enough feedback from real users be processed. Before that time, they made decisions based on opinions and assumptions. If they then make choices for a few improvements in mid-November, it will take another ten months before they have learned again. That is why it is important that the management team focus on shortening the T2L. By implementing improvements in a few places, the T2L can be lowered relatively quickly. Some examples are as follows:
By choosing a smaller customer group instead of solving everything for everyone, the construction step becomes significantly shorter (see Section 3.4 for more examples).
Because the solution is less extensive, feedback from users can often be collected more quickly.
By placing focus across departments and, in particular, reducing idle and waiting times, the turnaround time is considerably reduced.
In the preceding example, the T2L can be shortened relatively easily to five months.
Higher Agility Through Shorter T2L
Companies that have a long T2L will receive feedback from users about their strategic choices months or even years later. Companies with a short T2L can have that much earlier. The higher the learning speed, the faster companies can respond to developments. So the shorter the T2L, the higher the strategic agility of the organization. The agility of an organization is determined by the time it takes to change direction plus the time required to know whether this is a good change. Without the latter, the organization will change direction randomly and blindly. Agile companies are able to change direction quickly and then quickly get feedback on the new course.
T2L in Practice
How can the T2L be taken into practice? First of all, by including this standard in the various weekly and monthly reports of, for example, projects, improvement initiatives, team overviews, and products. This creates much more focus on improving the learning speed.
For this, it is useful to steer on a low T2L when making the plans. By choosing a phased rollout and not doing everything at once, the T2L is already considerably lower. In addition, it is a good indication of the risk for projects: a low T2L means a lower risk. There is more feedback from real users, which makes it clear more quickly if a project is feasible and valuable.
What is a good relationship between the duration of a project, initiative or improvement, and the corresponding T2L? For a project with an expected lead time of twelve months, a T2L of three months is pleasant. The shorter the better. For a project with a duration of four months, a T2L of one month is a good start. A T2L of one-fourth the lead time is often a good rule of thumb. In practice, medium-sized companies initially often have a T2L from nine months to more than a year. Before they start measuring and improving the T2L, these are recognizable numbers. Companies that have already made structural improvements for some time have a T2L of two to four months. Highly mature agile teams that issue releases to users several times a week have a T2L of less than one month.
It can be concluded that with the T2L, the learning speed of teams can be measured. This figure indicates how quickly agile teams learn. Lowering the T2L increases the learning speed. A lower T2L is therefore an improvement. The T2L is a handy tool that allows the agile leader to have control over the organization. It is also a good indicator to help teams know quickly whether they are doing the right thing. By including the T2L in matters such as reports, improvement initiatives, and projects, it is also applied and used in practice. It is the agile leader’s job to make the T2L transparent and to ensure that it is improved.
Steering on the T2L has four additional advantages that emphasizes the importance of a good T2L.
Four Additional Benefits of a Better T2L
There are four additional benefits of a better T2L.
Advantage 1: Lower Cost-of-Delay
If an analyst or programmer continues his morning work after lunch, he only needs a few minutes to get back to the topic. On the other hand, if he picks up work from a few weeks ago, it takes much more time to dive into the topic and recall all of the details. So, the more time that passes before he resumes work, the longer it takes for him to do so. For thought work, creative tasks, and unique activities, this is a latent cost that managers aren’t aware of; it’s the cost-of-delay. The longer things are paused or idle, the more expensive it becomes to pick them up again. The problem with this cost-of-delay is that it can hardly be measured. Indirectly, measurement can be achieved by measuring the difference between lead time and time spent. If something takes forty hours to make but it has taken two months, there is probably a significant cost-of-delay.
Lowering the T2L therefore automatically lowers the cost-of-delay. Work is quickly delivered to customers and quickly learned from, and therefore improvements can be made quickly. The employees involved need to delve much less often into topics in which they were engaged a while ago. They therefore have a much lower cost-of-delay.
Advantage 2: Lower Cost-of-Change
“We have already invested so much, we better continue.” “Couldn’t you say that earlier? Then I would not have had to redo this all over again!” “Now that I know this, a simpler solution would have also worked.” These are familiar phrases. As soon as months have been invested in a new product or expansion, stopping is more difficult than when much less effort has been invested. After all, if something has to be built in a different way, both the current way has to be taken out and the new way brought in. We call this double cost the cost-of-change (change costs). If a decision is made to change a product completely after just three months, very different amounts will be involved than if this is not decided until after ten months. That is why a better T2L also results in a lower cost-of-change.
Advantage 3: Higher Team Motivation
A team worked intensively together last week to make delivery possible. At the start of this week, they received enthusiastic feedback from users. This was very exciting. Imagine another team that got this feedback months later. This results in demotivation or passiveness. An improved T2L benefits motivation and cooperation in the team. They already have ideas for the next delivery. In addition, customers now see the results of their feedback in the new increment more quickly. Their ideas have been incorporated in the functionalities and services. As a result, the cooperation with customers has improved considerably, and the team receives much more valuable feedback. A lower T2L contributes to increased team motivation.
Advantage 4: Improved Liquidity
A T2L that is too long can be compared to too large a stock in a factory: it results in lower liquidity. The items have already been bought but have not yet been sold. As a result, there is a lot of money in stock at the warehouse, which could have been in the bank. The same applies to the work of teams that have not yet been sold. Employees have already spent a lot of time creating or adapting new products and services, but these are still in the warehouse, as it were. The labor has not yet been sold. The T2L gives a good indication of this stock of unsold labor. By multiplying the T2L with the internal cost price, the stock becomes transparent. The T2L indicates the moment when we received feedback from customers and learned from the time spent. So by lowering the T2L, liquidity can be increased because the time when products and services generate money is brought forward.
In the next section, I explain how teams can make their work more transparent every day and thus learn even more quickly.