Chapter 1. Introducing Application Lifecycle Management Methodology
This chapter introduces application lifecycle management (ALM) process and explains what you need to know in order to define an ALM that will help you implement a comprehensive and effective software or systems lifecycle. We discuss how to implement the ALM using agile principles in a real-world pragmatic way that will help guide the activities of each member of your team, whether you are creating new software or customizing a commercial package. Systems lifecycles are a little different than a software development lifecycle and are usually associated with obtaining (and customizing) a project from a solution vendor. Commercial off-the-shelf (COTS) software is commonly used today to deliver robust technology solutions, but they often require some effort to customize and implement. This is precisely when you might use a system development lifecycle instead of a software development lifecycle, which we discuss further in Chapter 2, “Defining the Software Development Process.” Both software and systems development require ALM, to help track and communicate required tasks and deliverables. An effective ALM must be, by its very nature, comprehensive, specific, and traceable. It is not easy to design a software lifecycle upfront, and we will guide you in accomplishing this effort in an agile and iterative way. This is not your typical agile book. We are going to have a difficult discussion about the most pragmatic approach to defining processes that are practical and yet comprehensive. Our approach to this work is very broad, and we touch on functions that are not normally thought of as being part of the ALM, such as internal audit and procurement. You will be reading about many different processes and functions. Try to ask yourself if this process or function is something that you need now or perhaps should be implemented in six to nine months as you begin to reach your first deadlines. You probably don’t need every single function or process that we discuss in this book, and that is fine. Our goal is to help you choose what your ALM will include in a practical and realistic way. We take this broad and comprehensive approach because the reality is that organizations we have assisted needed to design processes that aligned with the key functions and processes throughout the entire organization. We are going to talk about the “elephant in the room,” which is our reference to the fact that many companies are implementing processes using agile principles that are customized to their needs and often not completely agile in a textbook way. We are not trying to implement a purist approach to agile development. Instead, we are going to help you take your existing processes and make them better in a realistic and practical way. We have repeatedly seen that our colleagues are very smart and capable people who often make the tough choices required to optimize processes that are the best fit for their organization and the project that they are trying to complete. This book is about designing and implementing real-world processes that are customized to the needs of the organization and can evolve as requirements dictate. We are going to train that elephant and, more importantly, ensure that the team can enjoy open and honest communication. This chapter helps you understand application lifecycle management (ALM) and how to implement all of its features. In this chapter, we introduce the core concepts and then build upon them throughout the rest of the book.
1.1 Goals of Application Lifecycle Management
ALM helps you deliver your solutions faster and with better quality by providing much-needed structure for the entire software and systems development effort, including specific details on the tasks, roles, responsibilities, and essential milestones that help stakeholders track progress. ALM also helps us ascertain and communicate when we will be able to deliver the completed solution. Any parent taking a long trip with small children becomes accustomed to hearing the question, “Are we there yet?” Implementing an effective ALM is indeed a journey, and we need to be able to tell our stakeholders (in this case, my children) when we expect to arrive at the summer camp grounds or waterpark, which is our intended destination. In practice, the ALM essentially defines the rules of the road for the entire lifecycle. This is not a trivial effort, as even a small software or systems development effort involves many tasks, and managing the ALM can be an overwhelming task. It is also not a static endeavor, and any attempt to plan what you do not understand will obviously fail.
This chapter begins the process of providing a structure to enable you to manage the ALM along with each of the associated tasks that must be completed to implement a software or systems lifecycle. In Chapter 7, “Automating the Agile ALM,” we discuss the use of workflow automation to organize the tasks assigned to each stakeholder, and in Chapter 5, “Rapid Iterative Development,” we discuss the iterative nature of the ALM. The fundamental goal of application lifecycle management is to provide an appropriate structure to organize and manage the entire software and system development effort. The ALM helps each stakeholder understand what needs to get done on a day-to-day basis. Without a well-designed ALM, your software and systems development effort will be unstructured and most likely will fail. Let’s consider why the ALM is so important.
1.2 Why Is ALM Important?
Application lifecycle management is important because it provides the structure to understand and manage the myriad of complexities that are inherent in coordinating the work of many stakeholders. Small projects may only involve 10 or 20 technology professionals, including business analysts, software developers, quality assurance (QA)/testing engineers, project managers, operations, systems administrators, and, of course, our end users. However, larger projects can involve hundreds or even thousands of stakeholders. Providing clarity to the complex tasks, goals, and objectives of each contributor is essential for any project’s success and the success of the organization itself. The ALM is also important because it helps provide guidance to everyone involved on exactly what tasks they should be working on each day. This guidance is particularly important because large-scale development efforts always have a period that can only be described as “controlled chaos.” This situation isn’t necessarily unexpected, so long as you have the structure in place to help guide the effort. Providing this structure is precisely where the ALM can help.
Things can get very busy when you are part of a large software or systems delivery effort. Amidst the noise and “controlled chaos,” each member of the team needs to have a clear picture of their deliverables on a day-to-day basis. The ALM can provide this clarity through workflow automation, which we describe more fully in Chapter 7. The ALM answers the essential question of “are we there yet” by clarifying the tasks that have been completed and the tasks that remain. More importantly, the agile ALM, supported by workflow automation, proactively notifies stakeholders of pending tasks and escalates open items that have missed their planned deliverable date. Project management is certainly an essential function in any development effort, and if you would like to take a deep dive into agile project management then I recommend The Software Project Manager’s Bridge to Agility by Michele Sliger and Stacia Broderick.
The ALM does focus on effective release and iteration planning, thus helping to provide structure around what is often short-term planning that is typically very fluid and, at times, unpredictable. Application lifecycle management helps deal with these shifting priorities by clarifying the resources required for any effort and identifying impediments that could affect the project schedule or any of the deliverables. We discuss technical risk in Section 5.8 from the development view, which can significantly affect project delivery estimates.
ALM is fundamentally different from just a software or systems delivery lifecycle in that it casts a much wider net and can affect almost every function within the organization from the help desk, to the QA testers, to the operations team helping us stay online. Because of this wide scope, the ALM can help clarify connections between functions and helps us understand the big picture. Having a comprehensive approach to application lifecycle management makes the difference between successful organizations and those who risk failure. With this definition of ALM, we understand that the scope is indeed very wide, so the next question is: Where do we start?
1.3 Where Do I Start?
Getting started with an ALM effort involves understanding the business needs and requirements and defining the initial tasks that need to be completed. This effort can be very different based upon the business sector, outside pressures, and internal requirements. We have participated in organizations rushing to get a release to market in order to maintain (or establish) a competitive advantage. Other organizations are obliged to take a much more measured approach. Regardless, getting started usually involves identifying a reasonable pilot and communication with all stakeholders. We usually start by defining a very specific and minimal set of functionality that can truly add value, and we strongly defend against expanding the initial scope as much as possible. Getting something to work right as soon as possible has many benefits. The first is that you have something tangible to show and use to promote further collaboration. The second is that when you are first getting started, you are often dealing with new technology that must be understood, and this is best accomplished via bite-sized efforts. One good way to look at this effort is to consider how to get started with rolling out the tools necessary to support the ALM, and it should not come as a surprise that those of us who support application lifecycle management need to adhere to the same principles and approaches that we are advocating for our colleagues creating business applications. When implementing ALM tools, I like to get something running as quickly as possible. There are several reasons why I have found that what amounts to being an iterative approach works best.
Getting something up and running helps you clarify the vision and technology. This means that getting started often involves getting the iteration completed and shared with all of the relevant stakeholders.
With any new technology, there is a temptation to try to customize the solution to add features and make the product as productive to use as possible. Although this is obviously a valuable long-term goal, it is also hard to predict how the tool will actually be used—until it is actually in use. We try to get our ALM tools in limited use as soon as possible while continuously teaching and communicating what ALM is really all about.
1.4 What Is Application Lifecycle Management?
Application lifecycle management defines the rules of the road for the entire software and systems lifecycle. Successful ALM provides clarity around the entire delivery effort, from defining requirements to building, packaging, and deploying the code. The ALM can be considered from many viewpoints. Developers need clarity regarding which components they will work with and how they should be structuring their code. Operations needs enough lead time to create the infrastructure, along with estimates for required resources such as memory and disk space. A fundamental consideration is how developers will utilize source code management and workflow automation to achieve structure and clarity around their work. This may include defining how they use streams and components or perhaps a simpler branching methodology, which we discuss further in Chapter 6, “Build Engineering in the ALM.” They also need to define how they will handle application builds, continuous integration, and deployment into the initial test environments. The ALM inherently helps us to iteratively define and communicate what we are doing. Of course, the development team consists of many more stakeholders than just developers. For example, the relationship between development and testing should be a fundamental consideration in the ALM.
The QA and testing teams need to be able to understand the requirements and intended use of the system. Requirements management is an essential aspect of application lifecycle methodology, and QA needs to ensure that requirements are traceable and testable. The good news is that the ALM helps us to do this work. Each stakeholder has what can be a very different perspective, and the ALM must be flexible enough to define the work that is being done by each team and help manage change. Good planning does not prevent change. On the contrary, good planning helps to identify, manage change, and most significantly, assess and manage the impact of any changes that are introduced to the plan. Application lifecycle management helps clarify the roles and responsibilities of each of the stakeholders and then proceeds to provide guidance on what each member of the team should be doing on a day-to-day basis.
The broad scope of application lifecycle management does incur the risk that it can be too high level and frankly less than useful. Actually, it is our view that effective application lifecycle management should be very specific and detailed, with a practical and applied development methodology. In practice, we usually cannot implement the entire ALM at once. We generally focus on a specific aspect of the process that needs to be improved. This allows us to approach process improvement by creating a series of building blocks. To understand how to implement ALM, it can be helpful to look at the software development lifecycle (SDLC).
1.4.1 Remember the SDLC?
I remember the first time that I got to work on writing a detailed SDLC. It was exciting to brainstorm with colleagues on the tasks that we needed to complete for a successful software development effort. Back then I was a young computer programmer with a few years of experience and a strong interest in industrial psychology. I analyzed the tasks that were completed in each phase of the lifecycle with the same fervor that I imagined Winslow Taylor did in his famous time and motion studies.1
1. Taylor, Winslow, Scientific Management, 1911.
Now software development is certainly quite different from machine shop work in a factory. But, that said, there is considerable value in understanding what is involved with each phase of the software development lifecycle and providing that transparency to each of the stakeholders involved.
Creating an SDLC has typically been described in a waterfall lifecycle, with requirements, design, development, and testing being the high-level phases. The software development lifecycle is intended to help define all of the tasks, deliverables, and milestones required to create and implement software systems, but, in practice, it often falls short, and my experience has been that in many organizations the book typically sits on someone’s shelf without actually being used. This is quite disappointing because a well-defined SDLC can provide considerable value. We discuss a strategy for defining and automating your SDLC in Chapter 7.
At a practical level, the software development lifecycle helps to clarify all of the tasks required as part of the development effort. It also helps to manage the inevitable expansion in scope.
Software development lifecycles usually start with defining requirements, which gives this methodology an initial focus on understanding the organizational priorities and business focus.
1.4.2 Business Focus
Application lifecycle management must be driven by a pragmatic understanding of the business and business requirements. Technology professionals are notoriously focused on the technical details and often find it difficult to look at the world from a business perspective. I have faced this challenge myself many times while tackling a tough technical problem; my focus is often deep down into the “weeds,” and it can be very difficult to pull myself back up and look at things from the point of view of the business end user. In keeping with the wide scope of the ALM, business focus and business requirements must drive the ALM. The ALM affects requirements in several important ways. The first consideration is to have a well-defined lifecycle, including version control for the requirements themselves.
Obviously, the approach that you take with any lifecycle is directly affected by the business. For example, ALM in a highly regulated environment is very different than what is required for an Internet startup. The requirements themselves benefit from a development lifecycle, and, of course, business and technical requirements should be documented and version-controlled as necessary. Even in an agile environment, your epics and stories need to be under version control, and you may need to manage variants such as the requirements related to a bugfix to address a serious issue such as a regulatory violation. Requirements often need to be under version control and an associated workflow to support a requirements development lifecycle. In some industries, you may be required to maintain traceability between requirements and the test cases designed to verify required functionality. This may sound like an onerous approach with unnecessary extra steps, or what many agile practitioners refer to as “ceremony.” I am not advocating any more process than is absolutely necessary to avoid mistakes and their consequences. Less is more, and taking a Lean approach is almost always the optimal approach. However, designing a nuclear power plant actually requires that you not only have adequate documentation, but also verify that the documented requirements are accurate and maintained. The documents in safety systems often become essential artifacts in and of themselves and require a lifecycle to control and document changes. There are many industries where documentation is considered essential.
The ALM helps to control the evolution of the requirements themselves. Iterative development results in release milestones that help ensure the requirements are understood at a technical level. It is very common for developers to be given very limited information and be expected to know about details in how the system should work that are neither clarified nor documented. The third aspect to consider is the effort to understand the business requirements by the folks in the business unit itself. Although this need is often not recognized, our business colleagues also benefit from the iterative nature of the ALM as they, too, struggle to understand their own requirements, especially in environments that are changing constantly due to competitive or perhaps regulatory requirements.
Addressing the Business Silo
The industry adoption of DevOps principles and practices is bringing into focus the need for better communication between development, QA, and operations. Information security (InfoSec) is also a key stakeholder with a need for better communication and collaboration. Agile does put a much-needed focus on the role of the product owner, including colocation of the product owner with the development team. This is all well and good, but it has been my experience that many organizations often have a silo mentality between each of the technology teams and the business end user. Just as development needs to collaborate early and often with operations, so do the business minds need parameters to guide them in terms of documenting and clarifying business requirements. The business also helps us understand just how much change can be consumed by the end user. In fact, DevOps principles and practices should be used to help improve communication and collaboration between any silos within the organization. One key area to consider is risk and risk management.
Understanding Risk from a Business Focus
Risk is not inherently bad, and some businesses thrive on well-defined and accurately measured risk. I have worked in many trading firms that faced significant risk each and every day. Traders would make excellent decisions that resulted in substantial profits and then the next day, they would make the wrong choice and lose some money. The successful traders had made more good bets than bad bets and made lots of money. There are several important considerations from an ALM perspective. The first is to understand the risk as it relates to business requirements and then align accordingly. If you are in a high-risk business, then taking some well-measured risks might be quite acceptable—even desirable. The second consideration is that the ALM can help the business manage risk by driving technology to provide new services and features that help run the business. The ALM helps manage risk by providing a comprehensive framework to manage the full application lifecycle, especially application build, package, and deployment.
Risk is an essential issue to address and requires an open environment where all stakeholders feel safe expressing their views and opinions. There are many companies where the culture prevents employees from expressing their views, even punishing whistle-blowers who speak up when they see potential issues. Obviously, senior management may have more information and may be right when overruling an employee expressing a view from the trenches. But when the culture rewards silence and obstructs the free exchange of ideas, risk can often be a significant issue with severe incidents and systems outages as a consequence.
Agile practices are very effective, and we speak about them at length in this book. But there are times when you must use non-agile methodologies.
Agile or not, the ALM is an essential part of your software development methodology.
1.4.3 Agile or Not?
Many organizations have project and development management requirements that may affect the ALM. We have often seen that approval for the budget of a project required a waterfall or hybrid agile-waterfall approach where requirements, design, and a project plan were developed to a degree that management felt comfortable approving the budget. Many agile experts insist that the organization completely embrace agile principles and methodology. Although this may be a worthwhile goal, it is often not possible in many organizations. Even if you are in an organization that stubbornly resists becoming fully agile, you can often make your processes a little bit better by embracing agile methods. For some, what I am saying may seem like heresy, but it has been my experience that many companies are successfully employing a hybrid of agile and non-agile methods. There are reasons why this approach may be less than optimal and obviously it carries its own set of risks. But the “elephant in the room” is that many organizations must approach process improvement in an agile iterative fashion, and it is not always possible to just change the entire organization overnight. We discuss this issue further in Chapter 4, “Agile Process Maturity.” Whether your organization embraces agile or not, you can improve your development processes by adopting these industry best practices and the ALM is the key to helping you choose the right path and set the best possible priorities. We discuss the agile ALM in Chapter 3, “Agile Application Lifecycle Management.”
The degree of agility does affect how you implement your ALM. Another important consideration is the degree to which your process is mature or in a developing or more fluid state.
1.4.4 Mature Process or Fluid?
Process maturity is an important topic. We discuss agile process maturity in Chapter 4, but your first consideration is really whether or not your organization is ready for a mature ALM process or needs to stay fluid for a while. I try to avoid adding too much process in the beginning of any effort. It has been my experience that process has to evolve iteratively as the need for rules and guidelines organically takes shape. The reason for this is that implementing what can be perceived as too much process will motivate many members of the team to block your advance forward to really having a robust ALM. But catch the team after a systems outage where everyone had to stay up all night and you will find much more support for establishing IT controls to avoid mistakes. Another great motivator for process improvement is the need to pass an audit or meet federal regulatory requirements. Management is often motivated by an audit report or, even worse, a visit from a regulatory authority. This is precisely when you just might get the budget that you need for tools and consulting to help your organization transition to a more effective way of developing systems. It is also essential for process improvement to actually follow agile principles.
Is Your Process Improvement Agile?
Becoming agile is not just for the members of the scrum. I usually go into an organization with an existing set of practices, which may or may not actually be well-defined processes. My phone rings (or I get an email) when they need help. I approach the process improvement effort using the same principles that I am trying to implement. We discuss applying the principles and implementing the ALM in Chapter 4. In most cases, I must approach my work in an iterative fashion. I approach process improvement in an iterative way, and I encourage the teams that I am working with to approach their development effort in an iterative way too. There is a common misunderstanding among some folks that agile and iterative development are synonymous terms. They are not. Agile involves a specific set of principles and practices and is especially focused on the team transforming the way that they work. We discuss this further in Chapter 4, but I also want to point out that there are iterative development methods besides agile, and the ALM does facilitate iterative development whether the team is using agile methods or not.
1.4.5 Rapid Iterative Development
We discuss the iterative nature of the SDLC in Chapter 2 and the iterative waterfall in Chapter 14, “Agile in a Non-Agile World.” Whether you are embracing agile or not, the ALM facilitates rapid iterative development, which has long been regarded as an effective software methodology in and of itself. But iterative development is not necessarily agile. Rapid iterative development involves creating runnable versions of the system that can be used to clarify requirements and reduce risk in many important ways. The ALM benefits from an iterative approach and depends upon excellent configuration management best practices.
1.4.6 Core Configuration Management Best Practices
Configuration management best practices [1] are essential for implementing agile application lifecycle and DevOps. Without fully understanding these processes and practices, you just won’t get off home plate.
Source Code Management
Source code management is the discipline of safeguarding all of the artifacts that are created to develop your system. Source code management is a key function in configuration management (CM) and directly affects the productivity of your team, as well as the quality of the product being developed. Unfortunately, many organizations overlook the importance of establishing an effective source code management function to implement and support source code management tools and processes. You do not want to make this mistake, and in this chapter, I help you get started in the right direction. I have enjoyed having companywide responsibility for source code management in several large globally distributed organizations. This meant that I had to guarantee that the firm never lost any source code once it was part of a release to production (or even QA). SCM needs to be approached in a flexible and creative way. One size does not necessarily fit all, and I have often implemented source code management differently for some development teams than others. But I always focused on meeting the same essential goals as discussed later. I would also say that source code management provides the foundation for the other disciplines of configuration management, especially build engineering, release management, and deployment.
Good source code management starts with making certain all your source code is safely locked down and no important source code (or any other configuration item) is lost. That sounds fairly simple, but many of us recall the massive Y2K efforts that uncovered a remarkable number of critical systems that had been running in production for years without anyone knowing where the source code was located. In some cases, we went searching for the correct version of COBOL copybooks; in other cases, we had to rewrite the entire system from scratch (which was often the right choice anyway). My goal in implementing good Source Code Management is to absolutely guarantee that source code can never be lost.
Another important goal of effective code management is to help improve the productivity of your entire team. Effective Source Code Management means that you can manage more than one line of code development at the same time. It also means that you can improve the quality of your code in many ways, including helping to implement automated testing on both a unit and systems level. One of the most important goals of code management is to provide complete traceability so that you know exactly who changed your code and are able to, if necessary, back out the change. We discuss strategies for implementing effective source code management in our book Configuration Management Best Practices: Practical Methods that Work in the Real World.
Build Engineering
The goal of build engineering is to be able to reliably compile and link your source code into a binary executable in the shortest possible time. Build engineering includes identifying the exact compile and runtime dependencies, as well as any other specific technical requirements, including compiler (linker and managed environment) switches and dependencies. Build engineering improves both quality and productivity for the entire team. I believe that the build engineering team should consider themselves to be a service function with the development team as their primary customers. However, there are times when build engineering must also have the authority to enforce organizational policies. As build engineers, we provide a service to support the development effort, but our primary goal is to help secure the assets of the firm that are built and released through the build engineering function. We discuss build engineering in Chapter 6.
Environment Management
The goal of environment configuration is to always point to the correct runtime resources such as the QA or production database. Environment configuration is all about knowing your interface dependencies and controlling their changes accurately. Done well, environment configuration improves quality and productivity. Done poorly, environment configuration will be the cause of defects, wasted time, and other problems. Ultimately, the goal of environment configuration is establishing and maintaining control as your deliverable code makes its way from development to QA to production. Environment management also has the goal of providing enough test environments so that development can be done efficiently and with the proper utilization of available resources. Best practices in environment configuration will help you develop code faster and with better quality. We discuss environment management in Chapter 7, specifically Section 7.18.
Change Management
Change control is the most central function in configuration management. It is also one of the most underutilized and often misunderstood functions. Large organizations always have a change control board to act as a gatekeeper to control changes to the production environment. This is obviously important, but there is a lot more to change control than just gatekeeping. Some organizations have change control processes to govern requests for change. This is called a priori change control. Environment configuration is also commonly controlled by change control. I have seen organizations where there was a robust change control structure arranged in a hierarchy from senior management to the technicians making configuration changes and other organizations where the project managers and development team leads handled change control as an implicit project management task.
The goal of change control is to carefully manage all changes to the production (and usually QA) environments. Part of this effort is just coordination, and that is very important. But part of this is also managing changes to the environment that will affect all of the systems in the environment. It is also essential to control which releases are promoted to QA and production. Change control can act as the stimulus to all other configuration management–related functions as well. We discuss change management in Chapter 10, “Change Management.”
Release Management
Release management is a core function in configuration management that focuses on packaging a system for promotion from development to QA to production. If you are supporting a software production company, then “production,” for you, may be shipping the product to the customer instead of releasing the code to the production (or QA) environment. Although release management should focus on packaging the code created during the build process, in practice, release management is often viewed as being a very broad function that may encompass both source code management and build engineering. Release management in a corporate IT environment is slightly different from release management for a software product company—although I have worked in software product companies that still maintained separate QA, integration, and production environments as if they were a corporate IT environment, even while shipping the finished product to an end user (or pushing changes via an automated installation process). In this chapter, we focus on defining release management as a function that takes over after the build has been completed and prepares the release for deployment into the desired environment. Once a release has been created, it should conform to all of the standards set by the release management team.
The goal of release management is to create and maintain a repeatable process for packaging a release that includes a clear way to identify every component of the release. Release management must be clearly defined with little or no chance of errors occurring. Generally, release packaging is an automated function that includes creating an immutable ID that is embedded into the release package itself. Release management should also coordinate any dependencies that might be required in order for the release to successfully deploy. Finally, release management should be completely traceable with a clearly defined procedure to verify that the correct components have been deployed into a runtime environment.
Deployment
Deployment is the final step in the code promotion process. Deployment involves taking the packaged release and promoting it into the target environment, which is likely production or perhaps QA. Deployment is also responsible for rolling back a promotion if something goes wrong. If you are able to seamlessly deploy and roll back if necessary, you will be able to significantly reduce a common source of risk (and potential outages). Deployment is usually performed by the operations team, which often has a lot of other responsibilities that need to be fulfilled. I like to see deployment be the simplest step in the release process. This means that all of the prior steps of the process have been successfully completed. Deployment usually also means that control shifts from the release management team to the operations team, which is usually pretty busy with just maintaining the current production systems. The operations team is primarily concerned with keeping the systems running and responding to any events that occur. Therefore, deployment procedures should be kept as straightforward as possible. Your release management team needs to create solid deployment procedures that can be gracefully completed by the operations team. We discuss continuous deployment in Chapter 9, “Continuous Delivery and Deployment.”
The main goal of deployment is to promote a release into production without any possible problem occurring. Promoting a release should be like turning on a light switch. If there is a problem with the release, then the operations team follows the deployment procedures to roll back the release so that business can continue while your tech support team figures out how to solve the problem. I believe that the capability to gracefully roll back a release is just as critical a goal as promoting a new release into production. Deployment also has another important goal and that is to know exactly what is in production at all times and to know immediately if any unauthorized changes have been made. My primary goal in setting up any deployment process is that all changes are tracked and that promoting a release or rolling back is easy, reliable, and predictable.
1.4.7 Automation
Years ago, there was a common mantra that process was more important than tools, and that is certainly still true to a point. But the ALM is just about impossible to implement without effective tools and processes. Automation is fundamental in any useable ALM approach. The automation must include effective version control, automated application build, package, and deployment (including deployment frameworks). Automated code review is also important, along with unit and application testing. We discuss automation in Chapter 7, but suffice it to say that you just won’t have a very effective and viable ALM unless you embrace and implement automation. We discuss how to do that in this book. Among the most important practices in automation is continuous integration, which is a fundamental requirement for any effective ALM.
1.4.8 Continuous Integration
Continuous integration (CI) is an essential practice that involves putting together code that has been created and ascertaining if the code components can compile and run together successfully. Continuous integration often involves merging together code that has been written by different developers and is essential for code quality. We discuss continuous integration in Chapter 8, “Continuous Integration,” but want to note the fundamental value of this practice is that integrating a small amount of code as early as possible avoids much bigger issues later. It would be difficult to imagine an effective ALM that does not embrace integrating code frequently, although we also discuss a couple of situations where it is difficult or even impossible to achieve. This introduces risk, and just like any other risk, must be addressed. Similarly, continuous deployment is also a fundamental aspect of an effective ALM.
1.4.9 Continuous Deployment
Continuous deployment, much like continuous integration, involves handling the build, package, and deployment of a small amount of code instead of the much more risky approach of planning a big-bang deployment, many of which are infamous for taking everyone’s weekend. I have been doing DevOps and continuous deployment since long before it became a popular term in the industry. There has been considerable confusion between the terms continuous delivery and continuous deployment. Continuous delivery is typically used to indicate that you have a code baseline that is kept in a state where it could potentially be promoted to production at any time. In continuous delivery we sometimes promote changes to production but hide them using a technique called feature toggle until such time as we want the user to become aware of the new features. In the strictest sense, continuous deployment means that you are promoting changes all the way through to production on a continuous basis. For most large firms, continuous deployment may, at times, be impractical and often not even desirable, as many users do not like surprises, which can be quite disruptive. We discuss continuous deployment and continuous delivery in Chapter 9. Throughout this book we will use the term continuous deployment to generally mean promoting code to production as often as desired. Once an application is deployed, the operations team takes over and works to maintain continuous service without interruption.
1.4.10 Change Management
The goal of change management in the ALM is to carefully manage all changes to the production (and usually QA) deployment processes. Part of this effort is just coordination as well as communication, and that is very important. But part of this is also managing changes to the ALM that will affect all of the stakeholders in the organization. We discuss how to use change management throughout the ALM in Chapter 10.
1.4.11 IT Operations
The software development lifecycle typically does not extend beyond the application deployment. The ITIL v3 framework has become one of the most popular frameworks for defining IT service management. We discuss many of these best practices in Chapter 11, “IT Operations.” There is often a disconnect between operations and the development team, as each has a very different set of priorities. DevOps is emerging as a set of principles and practices intended to help improve communication and collaboration between development and operations.
1.4.12 DevOps
DevOps is a set of industry best practices that improve communication and cooperation between development and operations. We carefully examine the emergence of DevOps in Chapter 12, “DevOps,” and help you get started with implementing DevOps in your organization. This book takes a very wide view of DevOps and discusses DevOps principles and practices throughout the entire agile ALM.
1.4.13 Retrospectives
Retrospectives help drive the entire process improvement effort, especially in terms of defining application lifecycle management. This may not seem apparent at first, but your ALM must define the steps required to evaluate and solve problems. Retrospectives are very effective at enabling the team to understand what went well, as well as what needs to be improved. ALM should itself be defined in an agile iterative way, and that includes implementing retrospectives. Process improvement is hard. Identifying the processes supporting the application lifecycle is critical for your success. So how exactly do you go about implementing these best practices?
The most basic retrospective involves an open and honest discussion on what went well and what still remains to be improved. We dive into retrospectives in Chapter 13, “Retrospectives in the ALM.” Retrospectives also set the stage for determining what information should be communicated to senior-level management as part of IT governance.
1.4.14 IT Governance
IT governance provides transparency to senior management so that they can make the right decisions based upon accurate and up-to-date information. The ALM provides unique capabilities for ensuring that managers have the right information so that they can make reasoned decisions. From the CEO to the board of directors, information must often be compartmentalized due to the practical constraints of just how much information can be consumed at any point in time. Getting this right empowers your leadership to make the most well-informed decisions possible in order to steer the organization to success.
1.4.15 Audit and Regulatory Compliance
The goal of establishing IT controls is to support audit and compliance by implementing procedures that bring IT governance into alignment with corporate governance. The most classic case is to establish controls that ensure that financial reports are accurate. Compliance is usually implemented based upon an established standard or framework. This means that the IT controls established conform to the recommendations specified in the framework mandated by a regulatory agency or adopted by the organization. For example, most financial services firms are required to have a separation of duties between the physical control of assets, such as a payroll or banking system, and those who have responsibility for developing and maintaining these systems. They should also have appropriate security measures in place that prevent unauthorized access and, of course, modification. In the event of unauthorized access, detection and audit controls should make remediation possible. Audit and regulatory compliance help to identify and manage risk. Risk is not always negative, and many organizations thrive by taking calculated risks. But risk must always be identified with appropriate controls established to mitigate unavoidable risk. Regulatory compliance is most often associated with following specific federal laws, which actually often do provide effective guidance that helps reduce risk and safeguard the interests of all stakeholders.
1.4.16 ALM and the Cloud
Cloud-based computing promises—and often delivers—capabilities such as scalable, virtualized enterprise solutions, elastic infrastructures, robust services, and mature platforms. Cloud-based architecture presents the potential of limitless potential, but it also presents many challenges and risks. The scope of cloud-based computing ranges from cloud-based development tools to elastic infrastructures that make it possible for developers to use full-size test environments that are both inexpensive and easy to construct and tear down, as required. The first step is to understand how application lifecycle management functions within the cloud.
Technology professionals often use cloud-based tools at each stage of the development lifecycle to manage the workflow and all of its required tasks. Cloud-based ALM tools commonly include source code management, workflow automation (including defect and task tracking), knowledge management, and community-based forums. Cloud-based tools can be employed throughout the entire ALM. Many organizations make good use of the cloud by maintaining their own private cloud. Virtualized environments enable continuous delivery and robust testing environments. The real advantage of cloud-based computing is its ability to deliver enterprise architectures at low cost and then scale the architecture as demand increases. Companies that use cloud-based technologies can focus more on operating expenditures (OPEX) than on capital expenditures (CAPEX). Therefore, businesses can keep initial costs low and pay only for use of the resources as use of the system grows. Making the best possible choices requires that you understand the goals of the ALM in the cloud.
1.4.17 Mainframe
The mainframe is alive and well. Many corporations have been planning the deprecation of their mainframe infrastructure for years, and yet the mainframes continue to be in use. Application lifecycle management on the mainframe typically enjoys a very specific structure. The culture of the mainframe lends itself well to step-by-step defined procedures. Yet ALM on the mainframe often falls short of its potential. Sure, we can identify specific steps of a process, and everyone accepts that process tollgates are necessary on the mainframe. But that does not mean that our mainframe processes help improve productivity and quality. Even more essential, ALM on the mainframe must be agile and help the team reach their goals and the business achieve success.
1.4.18 Integration across the Enterprise
Understanding the ALM across the organization requires an understanding of the organization at a very broad level. It also requires that you understand how each structure within the company interfaces with the others. In DevOps, we call this systems thinking when we are examining an application from its inception to implementation, operation, and even its deprecation. DevOps principles and practices are essential in integrating the ALM across the organization.
1.4.19 Quality Assurance and Testing
Quality assurance and testing are essential to any software or systems lifecycle. Most technology professionals view the QA and testing process as simply executing test cases to verify and validate that requirements have been met and that the system functions as expected. But there is a lot more to QA and testing, and Chapter 21, “Personality and Agile ALM,” helps you understand how to establish effective processes to help ensure that your systems function as needed. DevOps helps us build, package, and deploy software much more quickly. Unfortunately, too often, the QA and testing process cannot keep up with the accelerated deployment pipeline.
1.4.20 Role of Personality
Technology professionals have remarkable analytical and technical skills. But even the most skilled professionals often have great difficulty dealing with the interesting behaviors and personalities of their colleagues. Implementing an agile ALM requires that you are able to work with all of the stakeholders and navigate all of the people issues inherent in dealing with diverse groups of often very smart people.
1.5 Conclusion
Agile ALM provides the essential structure to the entire software and systems development lifecycle. This includes specific details on the tasks, roles, responsibilities, and essential milestones that help us track progress. ALM also helps us ascertain and communicate when we will be able to deliver the completed solution. There are many ways to create an effective ALM. You need to start by understanding all of the inputs required by the ALM ecosystem. The rest of this book helps you understand how to design and implement an effective ALM. This book will also help you understand how to drive the entire ALM using DevOps principles and practices.
References
[1] Aiello, Bob and Leslie Sachs (2011). Configuration Management Best Practices: Practical Methods that Work in the Real World. Boston, MA: Addison-Wesley Professional.