Chapter 13. Retrospectives in the ALM
This chapter discusses the practical application of retrospectives to support application lifecycle management (ALM). The first section of this chapter will examine the main function of retrospectives, namely, to evaluate what went well and what needs to be improved. But that’s just the beginning. Getting accurate information from all stakeholders in a retrospective can be very challenging. If you are successful, the retrospective can help drive the entire ALM process. There are many possible modes for conducting a discussion, from in-person group meetings to online virtual conferences. You should consider the many views represented by each of the stakeholders from developers to business end users. Ultimately, you need to have a cross-functional view, and the retrospective may take on a very different approach depending upon the culture of the organization and the stakeholders involved.
Retrospectives require leadership, and this chapter will provide guidance on how to succeed if you are responsible for implementing this function. This includes walking the fine line between gently questioning and stronger probing when necessary. Understanding retrospectives from a cultural perspective is also essential. Fundamentally, retrospectives are an important aspect of process improvement leadership. We will discuss how to employ retrospectives to support ITIL incidents and problem management, along with other industry standards and frameworks. Crisis and risk management are also key considerations along with IT governance and compliance. Retrospectives take on a different tone when used to manage vendor performance. We will complete this chapter by considering how much process is necessary, how to deal with politics (or, more accurately, relationships), and the use of effective metrics to drive the process improvement journey.
13.1 Goals of Retrospectives
The goal of a retrospective is to provide a mechanism to facilitate feedback from all stakeholders, which will be incorporated into plans for improving your processes. Very few organizations spend enough time discussing specifically how they can improve. Our experience is that everyone usually feels relieved and excited once we conduct an effective retrospective, usually wondering why we don’t have this kind of feedback more often. The goal of retrospectives is to help you and your team improve your processes and procedures and ultimately achieve excellence in all you do.
(Author’s note: There are a few places where we found it easier to write in singular first person. Wherever you see “I,” you can assume that it is Bob speaking.)
13.2 Why Are Retrospectives Important?
Retrospectives are important because you don’t want to make the same mistakes over and over again. Retrospectives provide a much-welcomed framework for assessing what is being done well and what could be improved. We often find that people are amazed at how much information is gathered at a retrospective, and usually we hear people ask “Why didn’t we do this sooner?”
13.3 Where Do I Start?
Always begin by first asking participants to describe what went well. This discussion often elicits spontaneous remarks about what people feel could be improved. Ensure that everyone feels safe expressing their opinion and that even difficult topics can be comfortably discussed. You may need to structure your retrospectives based upon organizational dynamics and culture. For example, you might want to have a separate meeting among all of the DBAs (assuming they are a large group) and then have a DBA in the wider retrospective that includes representatives from development, operations, QA, and testing, as well as the business. Be careful with regard to stakeholders and their positional power. Many employees will not feel comfortable being open and candid in front of their direct manager. It is common to control participation to ensure that all those involved are able to speak freely and openly.
13.4 Retrospectives as Process Improvement
Retrospectives help drive the entire process improvement effort, especially in terms of defining application lifecycle management (ALM). This may not seem apparent at first, but your ALM must define the steps required to evaluate and solve problems. This is an ideal opportunity to identify processes and functions that should be added to your ALM. Retrospectives are effective at enabling the team to understand what went well and what needs active intervention to be improved. ALM should itself be defined in an agile iterative way, and that includes implementing retrospectives. Process improvement is hard. Identifying the processes supporting the application lifecycle is critical for your success. So, how exactly do you go about implementing these best practices?
The most basic retrospective involves an open and honest discussion on what went well and what needs to be improved.
13.4.1 Start with Assessing Success
Starting with the positive and upbeat discussion of what went well is often essential for a successful retrospective. Obviously, putting people on the defensive doesn’t motivate anyone to be open and forthcoming about things that could be improved. We have found that asking people about what went well often helps them to feel comfortable talking about what needs to be improved. It is also important to identify what went well for a number of reasons. The first is that you do not want to fix something that is not broken
Reinforcing what went well and what should be repeated is the first step. Next, the retrospective should dive deep into the issues and problems that occurred and what can be done to avoid repeating the same mistakes over and over again. Above all, it is essential that everyone participating in the retrospective feel safe expressing their views and observations in an open and honest way without fear of retaliation or punishment.
Understanding our mistakes is important for IT professionals—incidents and problems can point us in the right direction.
13.4.2 Incidents and Problems
Every development effort has its own share of challenges. Sometimes, these are minor issues that should be addressed (and hopefully are) in a timely manner. Sometimes, there are serious problems that require root-cause analysis. Taking a deep dive into analyzing why there was a problem and what we need to do in order to fix it will help avoid making the same mistakes time and time again.
Some people find it difficult to analyze these issues and problems and prefer to stay completely positive. These folks may appear to be uncomfortable having a candid, potentially contentious, conversation about what happened. This won’t help the team grow and improve. We are not suggesting that you embrace being negative either, but the ability to honestly and accurate self-appraise is actually a strength. In fact, a healthy and balanced view that truly reflects performance allows for the most constructive retrospectives and opens the door for further progress
This discussion must be conducted in a way that ensures everyone involved knows it is safe to have an open and honest discussion about mistakes that were made, while also examining the issues and problems that occurred. Remember, too, that mistakes can be good if their dissection yields concrete knowledge of how the process needs to be improved.
13.4.3 Mistakes Are Good
Mistakes are not always bad, and it is very important that everyone knows that they can talk honestly about mistakes that were made. Mistakes can actually be good for identifying important issues that need to be addressed. These discussions are not always fun, and obviously many people find it difficult to have an honest and open discussion about mistakes for which they could be held responsible. Some people find it easy to be straight and objective, and others may try to avoid the tough discussions at any cost. Personality traits may have a lot to do with why some people find these discussions to be difficult.
13.4.4 Personality and Disposition
Some people are just dispositionally dismal and negative. Others are consistently upbeat and may have difficulty discussing problems and incidents that occurred. Be aware that people with low self-esteem are particularly sensitive to criticism and blame situations that trigger their feelings of inadequacy. Ideally, you need to foster a culture where everyone knows that not only are they safe to have an honest and open discussion, but they will be valued and respected for any insights and observations that shed light on the issue at hand. We discuss personality in more detail in Chapter 21, but in the context of retrospectives, keep in mind that personality factors may bias some people’s participation. We see this behavior in people who just seem to always tell others what they believe that person wants to hear.
13.4.5 Don’t Just Tell Me What I Want to Hear
I sometimes find myself reminding people that doctors often have to deliver news that is difficult to accept and digest. That doesn’t mean that we don’t want to know the information. You would never want a doctor to spare you from learning that you have cancer—especially if there was a chance that aggressive treatment could alter the outcome. We might not enjoy hearing that we are sick, but the only way to get better is to get the message and information in a clear and straight manner.
The mode of delivery of sensitive information is also important. When people deliver sad news, we usually prefer to have the person come in and sit down. The mode of delivering information can obviously affect how the message is received and understood.
13.5 Which Mode Should You Use?
We all have different preferred modes of receiving information. Choosing the right mode of transmitting information is essential for your success.
13.5.1 In Person Is Best
Agile certainly places a premium on having all of the resources in one location, with face-to-face communication being the optimal approach. But is this true for everyone?
So, blind and deaf people aside, is it always easier to communicate face to face? For the most part it is, although we can recall another instance where a colleague of ours had a great deal of difficulty with communicating in English because he did not speak English as a first language. When we communicated face to face, we were generally confused and he was, too. Interestingly enough, he did much better when we used e-mail to communicate with each other. More and more organizations are finding it helpful to use online and video conferencing to communicate between teams.
13.5.2 Online and Video Conferencing
Conducting a retrospective online or through video conferencing can be quite effective. With online and video conferencing, you can usually see and hear all participants. The technology for online and video conferencing is noticeably getting better day by day, although we have seen challenges with the video picture updating on a delay and even sometimes poor voice quality. The technical quality of the connection can affect the quality of the retrospective because of the difficulty understanding the communication. Sometimes, you may find that it is necessary to rely upon audio-only teleconferencing.
13.5.3 Teleconference
The standard teleconference is nothing new and may be perfectly adequate for getting everyone to express their views and give input. The challenge is that you cannot see all of the participants and may not even know if they have muted their line and decided to have a side conversation with someone who is at their desk. I usually find it best to poll all of the participants by name on the call to ensure that you get their input—and their undivided attention. In the past, using virtual worlds and a personal avatar has enabled me to participate in meetings that I would otherwise have had to forego.
13.5.4 Virtual Worlds
Some of the software standards working groups that I have been involved with have taken to using virtual worlds for their meeting place. This provides some interesting capabilities and challenges. The virtual world can provide some helpful backdrops, including conference rooms and side discussions. In one instance, I had to leave the meeting early. I left my avatar online and came back later to review the conversation that had been recorded for my review. For all practical purposes, when I had to leave I had left a participant behind who carefully kept notes on the entire discussion. We have discussed some of the physical considerations inherent in conducting meetings, and most of us would prefer face-to-face meetings whenever possible. However, even when all parties are in the same room, you may find that there are a variety of different perspectives.
13.6 Perspective Is Essential
Each participant in the retrospective will likely have a unique perspective. That’s not to say that there won’t be a considerable number of shared views as well. Sometimes, the views are found to be distinguished by the function or role played by the participant. For example, developers often have a different view than customers.
13.6.1 Developers
In agile development, many developers play cross-functional roles that enable the technology professional to understand many different perspectives and views. That said, developers are still the folks charged with understanding the requirements and producing the system within a fixed time box (or sprint). Developers are often deep into the technology, confronting the many technical obstacles and forging solutions to deal with every possible challenge. Although the agile developer places a strong focus on the customer perspective, she or he still will likely have a different view than the end user.
13.6.2 Customers
The customer plays an essential role in agile development. Delivering the right features helps ensure that our company will be profitable and that we will all achieve success. The customer view should always be considered in the retrospective.
13.6.3 Tester
Testers also have a unique view, which focuses on ensuring that the system performs as specified and needed. We discuss QA and testing in Chapter 20, but in this chapter our focus is on using the retrospective to improve our processes, and the tester is a key resource in this effort.
13.6.4 Operations
Operations focuses on ensuring that systems are reliable and services are never interrupted. We discuss operations further in Chapter 11, but for this chapter you need to ensure that your operations team is well represented in all discussions on how to improve your processes.
Each of these perspectives is essential; the DevOps approach coaches us to include them all to yield a comprehensive and integrated cross-functional view.
13.7 DevOps: The Cross-Functional View
DevOps promotes the creation of a broad cross-functional view that allows us to understand each of the perspectives. This inclusive model is essential for conducting effective retrospectives with a holistic systems view.
13.8 Understanding the Use Case
Thoroughly understanding the use case points us to what we should test and how to actually perform the steps of the test.
13.8.1 Epics and Stories
Epics and stories describe the requirements of the system. This is also essential for us to discuss whether or not the system performed as expected and as needed.
13.9 Retrospectives as Leadership
Retrospectives provide a powerful opportunity to bring about positive change. Actualizing this potential often takes leadership and, especially, effective communication. Management needs to create an environment where each member of the team is comfortable expressing his or her own views without fear of retaliation. I have worked in a number of companies where many opportunities for improvement were missed because the team members did not feel comfortable expressing their views and opinions without fear of retaliation.
13.9.1 Removing Barriers
Retrospectives help us identify barriers that need to be addressed. In the scrum, the scrum master is responsible for ensuring that barriers are removed. The retrospective also helps identify barriers that are discovered as part of the review process.
13.10 Running the Meeting
Running the meeting should involve creating an agenda and setting expectations and sometimes entry criteria for starting the retrospective. If the meeting is going to focus on a specific incident or problem, then obviously those involved should have reviewed the details before the meeting begins. Additionally, someone should be the scribe to capture notes and action items that are agreed upon during the discussion.
13.10.1 Probing and Questioning
We like to use very nondirective probing when comments are not forthcoming to avoid influencing people’s responses. If this is not effective, then we may switch to being more direct with specific probes. Always remember that making everyone comfortable is of paramount importance if you are to get valid and useable data.
13.11 Retrospectives Supporting ITIL
Retrospectives should always be run in alignment with the culture of the organization. We find that many development teams are focused on agile development, whereas operations teams are often heavily focused on ITIL v3. It is always best if you can align the retrospective to match the terminology of your group. We sometimes find ourselves teaching agile principles to the operations team or explaining ITIL v3 terminology to developers, but aligning your approach to retrospectives will help your team achieve success.
13.11.1 Incidents
Retrospectives are often focused on understanding the circumstances in which an incident occurred. This approach can be effective at diving deep into how and why an incident occurred, as well as how it can be prevented in the future.
13.11.2 Problems
When the retrospective focuses on a problem, a wider group of stakeholders is usually involved and there is more of a focus on root-cause analysis. The ITIL v3 framework provides much-needed structure that can help ensure your retrospectives are effective and fruitful.
Retrospectives can also help triage defects.
13.12 Retrospectives and Defect Triage
When systems have challenges, we often find ourselves deluged with bug and defect reports. It can be challenging to prioritize and assign related work items to the limited stakeholders who have the knowledge of how best to address them. We have also seen many situations where even the subject matter experts (SMEs) were baffled. Holding a retrospective to triage bug and defect reports can be an excellent way to put some structure and priorities in place. Taking a DevOps cross-functional approach can also help by involving stakeholders from across the ALM in making the right decisions. The user or business expert can help set priorities where they may be difficult for developers to assess. The QA and testing group can create test cases to ensure that these defects do not come again in future releases, whereas the operations team may establish runbook procedures to work around systems issues in the future.
Understanding bugs and defects helps us address them, hopefully with bugfixes and features delivered quickly through our automated deployment pipeline. Having the entire team involved provides much-needed information and ensures that we are working toward ensuring the best possible service for our customers and other stakeholders.
13.13 Retrospectives as Crisis Management
We have seen some retrospectives that were held after a serious systems outage. In this case, we are not just trying to avoid the mistake again—we also might be discussing our response to the crisis itself. Life happens, and ensuring that your team always has an effective response ready and waiting is every bit as important as addressing the bugs and defects themselves. We have also seen situations where outages occurred due to circumstances completely beyond our control. Once again, the goal in this case is to ensure that our response itself is as effective as possible.
13.14 Supporting IT Governance
IT governance provides transparency to senior management, enabling them to make better decisions. We find that effective retrospectives can provide an excellent structure for gathering accurate information on what happened and what the team believes could help prevent issues from occurring again. It is common for the notes from a retrospective to be summarized and presented to senior management for their review.
13.15 Audit and Regulatory Compliance
Retrospectives often identify issues related to audit and regulatory compliance. Violating your own internal audit procedures can result in a serious impact, often involving senior management above you. When it involves regulatory compliance, sometimes federal laws may have been violated and the consequences can be even more severe. We have good news, though. It is almost always true that self-identifying the issue and coming up with a plan for addressing the incident can often help lessen the severity of any consequences. Although some regulatory violations have to be addressed by your legal and compliance department, in practice, identifying the issue and having a plan for avoiding the problem again in the future is most often an important step.
13.16 Retrospectives as Risk Management
Risk management is an important function in any organization, and retrospectives often provide important information to help ensure that risks are identified and an effective plan is created for addressing any possible contingencies.
13.17 Vendor Management
Vendors can be a source of serious challenges because they represent a factor that we cannot control. Having a retrospective when a vendor outage affects you can help identify strategies for addressing vendor-related risks and challenges. Sometimes, it also results in the decision to go and find another vendor, or at least avoid being locked into one who is not performing as needed.
13.18 Too Much Process
There is nothing worse than having too much process. When your team is mired in bureaucratic red tape, then everyone on the team will work hard to bypass your processes. When your retrospective identifies that you have too much process, it is clear that you should work with all of the relevant stakeholders to identify areas in which the process can be streamlined for increased efficiency and compliance.
13.19 Corporate Politics
Retrospectives sometimes become a discussion on corporate politics. Relationships are important, and understanding how things work in your company can certainly be a critical success factor, but try to prevent these discussions from just becoming a session to complain about things we cannot change. Although a little controlled venting can temporarily relieve frustration, coming up with a viable game plan that can really generate noticeable progress is a lot better.
13.20 Metrics and Measurement
It has been said that you can only improve what you can measure. Retrospectives are often a source of meaningful metrics that can help track gains as you improve your processes. Our advice here is to be careful that your metrics are valid and clear to all those involved. You want to avoid metrics that can be “gamed” by the stakeholders. For example, if my year-end bonus is based upon the number of bugs reported, then you might find my team fixing things without reporting them as bugs. Similarly, if my compensation is based upon the number of defects resolved, then you might find me reporting everything I can find just to show them as being resolved. Metrics need to be valid and reasonable.
13.21 Conclusion
Retrospectives can help you drive process improvement across the agile ALM. We view DevOps as being essential in implementing effective retrospectives because of its emphasis on the value of effective communication and collaboration across the enterprise. You need to implement effective retrospectives to successfully evolve your ALM over the course of your project.