The SDLC View

Let’s take a look at ALM from an SDLC perspective first. In Figure 2-3, you can see the different phases of a typical development project. Keep in mind that this is just a simplified view for the sake of this discussion. We have also tried to fit in the different roles from the ALM process presented earlier.

First, somebody comes up with an idea based on an analysis of the business needs: “Hey, wouldn’t it be great if we had a system that could help us do this (whatever the idea is)?” It could also be the other way around: the idea comes first, and the business value is evaluated based on the idea.

So an analysis or feasibility study is performed, costs are estimated, and hopefully a decision is made by IT and business management to start the project as an IT project. A project manager (PM) is selected to be responsible for the project and starts gathering requirements with the help of business analysts, PMO decision makers, and users or others affected. The PM also starts planning the project in as much detail as possible at this moment.

When that is done, the architect starts looking at how to realize the new system, and the initial design is chosen. The initial design is then evaluated and changed based on what happens in the project and what happens with requirements all through the project. After that, the development starts, including work performed by developers, user interface (UI) designers, and DBAs (and any other person not mentioned here but who is important for the project).

Testing is, at least for us, something done all along the way—from requirements specification to delivered code—so this not a separate box in Figure 2-2. After the system has gone through acceptance testing, it is delivered to operations for use in the organization. Of course it doesn’t end here. This cycle is most often repeated over and over again as new versions are rolled out and bug fixes implemented.

What ALM does in this development process is support the coordination of all development lifecycle activities from the preceding process through the following:

• Make sure we have processes that span these activities.
• Manage the relationships between development artifacts used or produced by these activities. (In other words, we talk about traceability here.)
• Reporting on progress of the development effort as a whole.

As you can see from this, ALM does not support a specific activity in itself. Its purpose is to keep all activities in sync. It does this just so we can focus on delivering systems that meet the needs and requirements of the business. By having an ALM process helping us synchronize our development activities, we can more easily see if any activity is underperforming and thus more easily take corrective actions.

The Service Management or Operations View

From a service management or operations view, we can look at ALM as in this quote from ITIL Application Management by the Office of Government Commerce in United Kingdom (TSO, 2002): ALM “focuses on the activities that are involved with the deployment, operation, support, and optimization of the application. The main objective is to ensure that the application, once built and deployed, can meet the service level that has been defined for it.”

When we see ALM from this perspective, it focuses on the life of an application or system in a production environment. If in the SDLC view the development lifecycle started with the decision to go ahead with the project, here it starts with deployment into the production environment. Once deployed, the application is operated by the operations crew. Bug fixes and change requests are handled by them and they also pat it on its back to make it feel good and run smoothly.

This is a quite healthy way of looking at ALM in our opinion, because we think that both development and operations are two pieces of ALM, cooperating in order to manage the whole ALM process. Both pieces are also something that should be thought of when planning a development project from the beginning; we cannot have one without the other.

The Application Portfolio Management View

The third view we will look at is the Application Portfolio Management (APM) view of ALM. In this view, we see the application as a product managed as part of a portfolio of products. We can say that APM is a subset of Project Portfolio Management (PPM), which we talked about in Chapter 1. Figure 2-4 describes this process.

This view comes from the Project Management Institute (PMI). Managing resources and the projects they work on is very important for any organization. In this view, we can see that the product lifecycle starts with a business plan—the product is the application or system that is one part of the business plan. An idea for an application is turned into a project and carried out through the project phases, until it is turned over to operations as a finished product.

When business requirements change or a new release (an upgrade in this figure) is required for some other reason, the project lifecycle starts again, and a new release is handed over to operations. After a while (maybe years) the system or application is discarded (this is called divestment, the opposite of investment). This view does not specifically speak about the operations part or the development part but should be seen in the light of APM instead.

The Unified View

The final view is a unified view of ALM. In this view, we have made an effort to align all these views with the business. Here we do as the CIO would do: we focus on the business needs, not on separate views. This we do to improve the capacity and agility of the project from start to end. Figure 2-5 shows an overview of how these three views are included in the whole unified ALM aspect of a business.

Traceability of Relationships Between Artifacts

Some customers we have seen have stopped doing upgrades on his systems that were running in production because the company had poor or even no traceability in its systems. For these customers it was far too expensive to do upgrades because of the unexpected effects even a small change could have. The company had no way of knowing which original requirements were implemented where in the applications. This customer claimed, and we have seen and heard this in discussions with many other customers, that traceability can be a major cost driver in any enterprise if not done correctly.

There must be a way of tracing the requirements all the way to delivered code—through architect models, design models, build scripts, unit tests, test cases, and so on—not only to make it easier to go back into the system when implementing bug fixes, but also to demonstrate that the system has delivered the things the business wanted.

Another reason for traceability is internal as well as external compliance with rules and regulations. If we develop applications for the medical industry, for example, we need to have compliance with FDA regulations. We also need to have traceability when change requests are coming in so that we know where we updated the system and in which version we performed the update.

Automation of High-Level Processes

The next pillar is automation of high-level processes. All organizations have processes, as you saw in Chapter 1. For example, there are approval processes to control hand-offs between the analysis and design or build steps, or between deployment and testing. Much of this is done manually in many projects, and ALM stresses the importance of automating these tasks for a more effective and less time-consuming process. Having an automated process also decreases the error rate compared to handling the process manually.

Visibility into the Progress of Development Efforts

The third and last pillar is providing visibility into the progress of development efforts. Many managers and stakeholders have limited visibility into the progress of our development projects. The visibility they have often comes from steering group meetings, during which the project manager reviews the current situation. Some would argue that this limitation is good, but if we want to have an effective process, we must ensure visibility.

Other interest groups such as project members also have limited visibility of the whole project despite being part of the project. This often comes from the fact that reporting is hard to do and often involves a lot of manual work. Daily status reports would quite simply take too much time and effort to produce, especially when we have information in many repositories.

Application Lifecycle Management 1.0

As software has become more and more complex, we have seen that role specialization has increased in IT organizations. This has led to functional silos in different areas (roles) such as project management, business analysis, architecture, development, database administration, testing, and so on. As you may recall from the beginning of this chapter, you can see this in the ALM circle. There is no problem with having these silos in a company, but having them without any positive interaction between them is.

There is always a problem when we build great and impenetrable walls around us. Most ALM vendors have driven the wall construction because most of their tools historically have been developed for particular silos. If we look at build management tools, they have supported the build silo (naturally) but have little or no interaction with test and validation tools (which is kind of strange since the first thing that usually happens in a test cycle is the build)—just to mention one area. This occurs despite the fact that interaction between roles can generate obvious synergies that great potential. We need to synchronize the ALM process to make the role-centric processes a part of the overall process. This might sound obvious, but has just not happened until lately.

Instead of having complete integration between the roles or disciplines mentioned in the beginning of the chapter and the tools they use, we have had point-to-point integration—for example, we could have a development tool slightly integrated with the testing tool (or probably the other way around). Each tool uses its own data repository, so traceability and reporting is hard to handle in such an environment as well (see Figure 2-7).

This point-to-point integration makes the ALM process fragile and quite expensive as well. Just imagine what happens when one tool is updated or replaced. Suddenly, the integration might break and new solutions have to be found to get it working again. This scenario can be reality if, for example, old functions in the updated or replaced tool are obsolete and the new one does not support backward compatibility. This can be hard to solve even with integration between just two tools. Imagine what happens if we have a more complex situation, including several more tools. We have seen projects using six or seven tools during development, creating a fragile solution when new versions have been released.

The tools have also been centered on one discipline. In real life, a project member working as a developer, for instance, quite often also acts as an architect or tester. Because the people in each of these disciplines have their own tool (or set of tools), the project member must use several tools and switch between them. It could also be that the task system is separated from the rest of the tools so to start working on a task, a developer must first retrieve the task from the task system—probably print it out, or copy and paste it, then open the requirements system to check the requirement, then look at the architecture in that system, and finally open the development tool to start working. Hopefully, the testing tools are integrated into the development tool; otherwise, yet another tool must be used. All this switching costs valuable time better put into solving the task.

Having multiple tools for each project member is obviously costly as well because they all need licenses for the tools they use. Even with open source tools that may be free of charge, we have maintenance costs, adaptions of the tools, developer costs, and so on. Maintenance can be very expensive, so we should not forget this even when the tools are free. So, such a scenario can be very costly and very complex. It will also most likely be fragile.

As an example, I have two co-workers working at a large medical company in Gothenburg. They have this mix of tools in their everyday work. I asked them to estimate how much time they needed to switch between tools and get information from one tool to another. They said they probably spend half an hour to an hour each day syncing their work. Most times they are on the lower end of this scale, but still in the long run this is a lot of time and money. My friends also experienced big problems whenever they needed to upgrade any of the systems they used.

One other problem with traditional ALM tools worth mentioning is that vendors often have added features, for example, adapting a test tool to support issue and defect management. And in the issue management system, some features might have been added to support testing. Because neither of the tools have enough features to support both disciplines, the users are confused and will not know which tool to use. In the end, most purchase both just to be safe, and end up with the integration issues described earlier.

Application Lifecycle Management 2.0

So let’s take a look at what the emerging tools and practices (including processes and methodologies) in ALM 2.0 try to do for us. ALM is a platform for the coordination and management of development activities, not a collection of lifecycle tools with locked-in and limited ALM features. Figure 2-8 and Table 2-1 summarize these efforts.

One of the first things we can see is a focus on plug-ins. This means that from one tool, we can add the features we need to perform the tasks we want. Without using several tools! If you have used Visual Studio, you have seen that it is quite straightforward to add new plug-ins into this development environment. Support for Windows Communication Foundation (WCF) and Windows Presentation Services, for example, was available as plug-ins long before their support was added as a part of Visual Studio 2008.

Having the plug-in option and making it easy for third-party vendors to write plug-ins for the tool greatly eases the integration problems discussed earlier. You can almost compare this to a smorgasbord, where you choose the things you want. So far this has mostly been adopted by development tool vendors such as IBM or Microsoft, but more are coming. Not much has happened outside of development tools so far but TFS offers some nice features that definitely will help us a lot.

Teamprise, a third-party vendor, developed a solution giving access to the Team Foundation Server (TFS) from a wide array of platforms, including Mac OS X (see Figure 2-9). In November 2009, Teamprise was acquired by Microsoft. After the acquisition TFS 2010 changed its name to Team Explorer Everywhere. In writing this book we used Team Explorer Everywhere on our Mac OSX laptops using the Eclipse development platform.

Another thing that eases development efforts is that vendors in ALM 2.0 are focusing more on identifying features common to multiple tools and integrating these into the ALM platform. We find things like the following among these:

• Collaboration
• Workflow
• Security
• Reporting and analysis

You will see later in Chapter 4 that TFS has these features embedded out of the box. Microsoft uses Microsoft Office SharePoint Server/Windows SharePoint Services as part of the TFS suite for collaboration among team members, for example. Active Directory is used for authentication and security. SQL Server obviously is the data repository giving us access to tools such as SQL Server Analysis Services for analytics and SQL Server Report Builder for reports. Microsoft has done a good job in giving us tools to get the job done.

Another goal in ALM 2.0 is that the tools should be repository neutral. They say that there should not be a single repository but many, so we are not required to use the storage solution that the vendor proposes. IBM, for example, has declared that their coming ALM solution will integrate with a wide variety of repositories, such as Concurrent Versions System (CVS) and Subversion, just to mention two. This approach removes the obstacle of gathering and synchronizing data, giving us easier access to progress reports, and so on. Microsoft uses an extensive set of web services and plug-ins to solve the same thing. They have one storage center (SQL Server), but by exposing functionality through the use of web services, they have made it fairly easy to connect to other tools as well.

An open and extensible ALM solution lets companies integrate their own choice of repository into the ALM tool. Both Microsoft and IBM have solutions—data warehouse adapters—that enable existing repositories to be tied into the ALM system. It is probable that a large organization that has already made investments in tools and repositories in the past doesn’t want to change everything for a new ALM system; hence it is essential to have this option.

Any way we choose to solve the problem will work, giving us possibilities of having a well-connected and synchronized ALM platform.

Furthermore, ALM 2.0 focuses on being built on an open integration standard. As you know, Microsoft exposes TFS functionality through web services. This is not publicly documented however so we need to do some research and trial and error before we can get this working. This way, we can support new tools as long as they also use an open standard; and third-party vendors have the option of writing tons of cool and productive tools for us.

Process support built in to the ALM platform is another important feature. By this I mean having the automated support for the ALM process built right into the tool(s). We can, for instance, have the development process (RUP, SCRUM, XP, and so on) automated in the tool, reminding us of each step in the process so that we don’t miss creating and maintaining any deliverables or checkpoints.

In the case of TFS, you will see that this support includes having the document structure, including templates for all documents, available on the project web site, directly after having created a new TFS project. We can also imagine a tool with built-in capabilities helping us in requirements gathering and specification, for instance—letting us add requirements and specs into the tool and have them transformed into tasks assigned to the correct role without having to do this manually.

An organization is not likely to scrap a way of working just because the new ALM tool says it cannot import that specific process. A lot of money has often been invested in developing a process, and an organization is not likely interested in spending the same amount again learning a new one. With ALM 2.0 it is possible to store the ALM process in a readable format such as XML.

The benefits include that the process can be easily modified, version controlled (you need to do this yourself, TFS doesn’t do it for you), and reported upon. The ALM platform can then import the process and execute the application development process descriptions in it. Microsoft uses XML to store the development process in TFS. In the process XML file the whole ALM process is described, and many different process files can coexist. This means we can choose which process template we want to base our project on when creating a new project (see more in Chapter 6).

As we saw earlier, it is important for an enterprise to have control over its project portfolio to better allocate and control resources. So far, none of the ALM vendors have integrated this support into the ALM platform. There may be good reasons for this though. For instance, while portfolio management may require data from ALM, the reverse is probably not the case.

The good thing is that having a standards-based platform makes integration with PPM tools a lot easier.

Application Lifecycle Management 2.0+

So far not all the ALM 2.0 features have been implemented by tools vendors. There are various reasons for this. One of these is the fact that it is not quite easy for any company to move to a single integrated suite, no matter how promising the benefits might look when you see them. To make such a switch would mean changing the way we work in our development processes and even in our company. Companies have invested in tools and practices and spending time and money on a new platform can require a lot more investment.

For Microsoft focused development organizations the switch might not be so hard however, at least not for the developers. They already use Visual Studio, SharePoint, and many other applications in their daily life and the switch is not that big. But Microsoft is not the only platform out there and competitors like IBM, Serena, and HP still have some work to do to convince the market.

We also can see that repository neutral standards and services have not evolved over time. Microsoft’s for instance still rely on SQL Server as a repository and have not built in much support for other databases or services. The same goes for most competition to TFS.

The growth of agile development and project management in recent years has also changed the way ALM must support development teams and organizations. We can see a clear change from requirements specs to backlog-driven work and the tooling we use needs to support this in a good way.

Agile practices such as build and test automation become critical for our ALM tools to support. Test Driven Development (TDD) continues to rise and more and more developers require their tools to support this way of working. If the tools don’t do that they will be of little use for an agile organization. Microsoft has really taken the agile way of working to their hearts in the development of TFS. We will show you all you need to know about the support for agile practices in TFS all through this book.

We can also see a move from traditional project management toward an agile view where the product owner and scrum master require support from the tools as well. Backlog grooming (the art of grooming our requirements in the agile world), agile estimation and planning, reporting,—important to these roles—need to be integrated to the overall ALM solution. We will come back to these concepts in Chapter 6.

The connection between operations and maintenance becomes more and more important. Our ALM tools should integrate with the tools used by these parts of the organization. Fortunately, Microsoft has plans for this. They plan to make the integration between MS Systems Operations Manager and TFS much simpler and built in out of the box, adding support for maintenance and operations.

In the report “The Time is right for ALM 2.0+” Forrester research presented the ALM 2.0+ concept as you can see in Figure 2-10. (Forrester Research, Inc., The Time Is Right For ALM 2.0+, October 19, 2010) In their report Forrester Research, Inc extended traditional ALM with what they called ALM 2.0+. Traditional ALM covers traceability, reporting, and process automation. Forrester Research, Inc envisions the future of ALM to also include collaboration and work planning. We like this idea and we will try to show you that Microsoft also has support for this in TFS.