© Joachim Rossberg 2019
Joachim RossbergAgile Project Management with Azure DevOpshttps://doi.org/10.1007/978-1-4842-4483-8_7

7. Metrics in Agile Projects

Joachim Rossberg1 
(1)
Kungsbacka, Sweden
 

A key performance indicator (KPI) is a performance measurement used in most organizations to evaluate an organization’s success or the success of a particular activity within the organization. Often, KPIs are used to measure the effects of a change project—for instance, implementing a good DevOps process—or to evaluate the progress of a development project.

You can use the score from a DevOps online assessment as a KPI and compare the assessment scores before and after the implementation of a DevOps process improvement. In this way, you get an indication of whether you have improved as a result of implementing a new process.

During projects, you should also be able to use the reports from your DevOps toolset to determine whether you’re constantly improving your work. Continuous improvement, in my opinion, is something to strive for. When it comes to project management, you can, for instance, look at the team’s velocity (how fast the team is able to work) and determine whether it’s increasing or decreasing. By using reports and metrics from your DevOps tools, you can choose the KPIs you want and learn how to evaluate them.

This chapter looks at metrics for five topics that cover most aspects of software development. Keep in mind they are not only for Agile projects, but can be used in many other projects as well:

  • Project management

  • Architecture, analysis, and design

  • Developer practices

  • Software testing

  • Release management

Keep one thing in mind while you read about some of the reports. Reports are not available in Azure DevOps—only in TFS 2018. For Azure DevOps, I encourage you to use charts instead to get the information you need. Charts are described later in this chapter.

Some reports require that the team project collection that contains your team project was provisioned with SQL Server Reporting Services. The report is not available if Reports does not appear when you open Team Explorer and expand your team project node.

Metrics for Project Management

To get good metrics about the status of your projects, it’s important to measure your progress. You can do this in several ways. If you’re using Agile as a methodology, many of these metrics and reports should be familiar. To others, they may be new. Keep in mind that not all of these reports are available in Azure DevOps, but only in TFS on-prem.

Agile Metrics

Let’s look at some important reports that are commonly used in Agile practices:

  • Backlog overview

  • Sprint burndown

  • Velocity report

  • Release burndown

  • Remaining work

  • Unplanned work

The backlog overview report lists all user stories, filtered by tags and iteration, and order of importance. Basically, this is a list of user stories filtered by the criteria you need. Many people use Excel (or another spreadsheet application) to create this report, but many DevOps tools have built-in support for producing it. Figure 7-1 shows what it looks like in Azure DevOps.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig1_HTML.jpg
Figure 7-1

The backlog overview in Azure DevOps

I’ve mentioned the sprint burndown report before (Figure 7-2). This report shows how much work there is left to do in a sprint. Using it, you can predict when the team will finish the work assigned to this sprint, either during the sprint or after the sprint is finished. Based on this information, the team and the product owner can take actions to make sure they deliver what they have committed to deliver.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig2_HTML.jpg
Figure 7-2

Sprint burndown report

The release burndown report (Figure 7-3) shows the same thing as the sprint burndown, but for the work included in a release.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig3_HTML.jpg
Figure 7-3

Release burndown report

A burndown and burn rate report (Figure 7-4) is another way to show a project’s burndown. No surprises here: This is the same information shown in Figure 7-1. The burn rate provides summaries for the completed and required rate of work for a specified time period. In some tools, you can also see the information for team members. You can sometimes choose to see the report based on hours worked or number of work items.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig4_HTML.jpg
Figure 7-4

Burndown and burn rate report

Velocity (how much work a team can take on in a sprint) is important, especially for a product owner planning how much work can be accomplished in coming sprints. Velocity is usually a measure of the effect per story point that the team can accomplish.

Before any work is started, the product owner calculates a theoretical velocity to begin planning. As time goes by, it’s updated with the team’s real velocity based on how much work it delivers in each sprint. This helps the product owner estimate how much work the team can take on in coming sprints. The velocity report (Figure 7-5) can help you retrieve this information easily. Here you see how much effort the team has delivered for each sprint.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig5_HTML.jpg
Figure 7-5

Velocity report

The remaining work report (Figure 7-6) is another useful metric. You can use it to track the team’s progress and identify any problems in the flow of work. With some tools, you can view this report in either an Hours-of-Work view or a Number-of-Work-Items view.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig6_HTML.jpg
Figure 7-6

Remaining work report

The unplanned work report (Figure 7-7) is useful when the team plans an iteration by identifying all work items it intends to resolve or close during the course of the iteration. Work items assigned to the iteration by the plan completion date of the report are considered planned work. All work items added to the iteration after that date are identified as unplanned work.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig7_HTML.jpg
Figure 7-7

Unplanned work report

Metrics for Architecture, Analysis, and Design

DevOps tools don’t include many metrics you can use for KPI assessment for architecture, but you can use some taken from the development area. Using code metrics, you can get information about how your architecture and design are working, including the following:

  • Lines of code: Presents an approximate number based on Intermediate Language (IL) code. A high count may indicate that a type or method is doing too much work and should be split up. This may also be a warning that code will be hard to maintain.

  • Class coupling: Measures the coupling to unique classes through parameters, local variables, return types, method calls, generic or template instantiations, base classes, interface implementations, fields defined on external types, and attribute decoration. Strive for low coupling. High coupling indicates a design that is difficult to reuse and maintain because of its many interdependencies on other types.

  • Depth of inheritance : Indicates the number of class definitions that extend to the root of the class hierarchy. The deeper the hierarchy, the more difficult it may be to understand where particular methods and fields are defined and/or redefined.

  • Cyclomatic complexity : Is determined by calculating the number of different code paths in the flow of the program. It indicates the code’s complexity. A high complexity makes maintainability suffer, and it can also be hard to get good code coverage.

  • Maintainability index: Is an index value between 0 and 100 that represents the relative ease of maintaining the code. The higher the better; a rating of more than 60 is good. Less than that, maintainability suffers.

Some DevOps tools can generate dependency graphs. These graphs are used to visualize code and its relationships. Running analyzers on these graphs can give you useful information as well:

  • Circular references are nodes that have circular dependencies on one another.

  • Hubs are nodes that are in the top 25% of highly connected nodes.

  • Unreferenced nodes have no references from any other nodes.

Using these analyzers, you can determine whether you have loops or circular dependencies so that you can simplify them or break the cycles. You also can determine whether you have too many dependencies, which could be a sign that they’re performing too many functions. To make the code easier to maintain, test, change, and perhaps reuse, you need to look into whether you should refactor these code areas to make them more defined. You may also be able to find code that performs similar functionality and merge with it. If the code has no dependencies, you should reconsider keeping it.

Metrics for Developer Practices

Metrics for developer practices are KPIs that can help you understand if you’re working successfully to improve your code. These measures are useful from both the architectural and design viewpoints as well as from a developer viewpoint. Using them helps you improve how you design your application or system.

Several important metrics are available automatically in many tools and can help you get a good understanding of the quality of your development work:

  • Code coverage

  • Code metrics

  • Compiler warnings

  • Code analysis warnings

Code Coverage

Code coverage shows you how much of the code has been covered by automated unit tests. You get the value as a percentage of the entire code base. The difficulty often is deciding what percentage is enough. Should you always strive for 100%? Or is 80% enough? This is something the team has to discuss with the product owner when using Scrum or a similar decision maker when using other processes. This value is input as the definition of done.

Code Metrics

You can look at several different code metrics. Lines of code, class coupling, depth of inheritance, cyclomatic complexity, and the maintainability index—as described in the earlier section “Metrics for Architecture, Analysis, and Design”—also apply to code metrics.

Compiler Warnings

Errors and warnings should be avoided in a project. Allowing more than zero errors or warnings tends to result in the team accepting lower quality in the code base, which over time causes the code to lose maintainability.

Track this metric to make sure the number of errors is zero. Ideally, this should be enforced by automatic build policies.

Code Analysis Warnings

Code analysis in development tools performs static analysis on code, which helps developers identify potential design, globalization, interoperability, performance, and security problems, to name a few. Much of this functionality is currently available only for .NET development; if you’re using Java, things may be different.

Code analysis tools provide warnings that indicate rule violations in managed code libraries. The warnings are organized into rule areas such as design, localization, performance, and security. Each warning signifies a violation of a code analysis rule.

Code analysis can be used to enforce company policies on the code developers write. Many DevOps tools offer good support for code analysis, and usually include a set of rules. Often, you can extend the functionality by writing your own rule set or by suppressing the rules you don’t want. Definitely discuss this analysis with your development team and the product owner, because the warnings have an impact on the effort required before the definition of done is fulfilled.

Metrics for Software Testing

Software testing is an important area. Testing should be a constant part of any development effort, not just a phase at the end of a project. There are good metrics you can use during your projects to make sure you have high-quality testing in place. The following are a number of metrics you can use as KPIs for software testing:

  • Number of bugs per state: This metric tells you how many bugs are active, resolved, or closed; whether the number of active bugs is increasing and whether the number of resolved and closed bugs is constant. If the numbers stay constant, you need to look into how you perform your testing.

  • Number of bugs sent back from testers for more information (aka “reactivated bugs”): A large number of reactivated bus may indicate that communication between developers and testers must improve.

  • Code coverage: This metric shows how much of the code has been covered by automated unit tests. You get the value as a percentage of the entire code base.

  • Test run results: This metric indicates how your tests performing, and whether you have many failed tests. If you do, you need to look at what can be done to improve the tests.

  • Percentage of requirements covered by test cases: As the title implies, this metric indicates the percentage of requirements covered by test cases.

  • Percentage of requirements covered by testing: Do you actually verify the requirements with the test cases you have? If this figure is low and the figure for percentage of requirements covered by test cases is high, you may have an issue you need to deal with.

Example Reports

The metrics you get in your reports concerning testing can be very helpful to your projects. The reports described here are found in many tools:

  • Bug status reports

  • Reactivations reports

  • Bug trend reports

Bug Status Report

The bug status report gives you information about the cumulative bug count based on bug state, priority, to whom the but is assigned, and bug severity. The document shows you the number of bugs and the number of resolved bugs (see Figures 7-8 and 7-9).
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig8_HTML.jpg
Figure 7-8

Bug status report

../images/477063_1_En_7_Chapter/477063_1_En_7_Fig9_HTML.jpg
Figure 7-9

Bug status report

Figure 7-8 shows the number of bugs over time. You can see how the numbers of active, closed, and resolved bugs change. In this case, the number of active bugs is decreasing and the number of closed and resolved bugs is increasing, leading to a point where the number of active bugs is zero.

Figure 7-9 shows a report that displays how many bugs are assigned to an individual user. You can also see the priority of each bug as well as how many bugs have been resolved by the users.

Reactivations Report

The reactivations report (Figure 7-10) is used to determine how many bugs have been resolved or closed too early. If a bug needs to be opened again, it’s called a reactivation. A high number indicates the developers need to improve their bug-fixing process and not close or resolve the bugs unless they really are ready to be closed. A high number may also be an indication that you have bad communication between testers and developers. For instance, incomplete test reports and poorly written test cases can lead to a greater number of reactivations.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig10_HTML.jpg
Figure 7-10

Reactivations report

Bug Trend Report

Next is the bug trend report (Figure 7-11). This report helps you track the rate at which your team is finding, resolving, and closing bugs.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig11_HTML.jpg
Figure 7-11

Bug trend report

Metrics for Release Management

A quick look at the ITIL ( www.itilnews.com/ITIL_v3_Suggested_Release_and_Deployment_KPIs.html ) will give you some other KPIs you can use. If you want to use them, you may need to create your own reports to automate the retrieval of this information. ITIL mentions these KPIs, among others:

  • Number of software defects in production [the number of bugs or software defects of applications (versions) that are in production]

  • Percentage of successful software upgrades (excludes full installations)

  • Number of untested releases (not tested and signed off)

  • Number of urgent releases

  • Average costs of release, for which costs are most likely based on man-hours spent

    Note ITIL is a set of practices for ITSM that focuses on aligning IT services with the needs of business. ITIL describes procedures, tasks, and checklists that aren’t organization specific but are used by organizations to establish a minimum level of competency. ITIL allows an organization to establish a baseline from which it can plan, implement, and measure. ITIL is used to demonstrate compliance and to measure improvement.

Example Reports

Following the progress of your builds is essential to keep track of quality. These build reports differ from DevOps platform to platform, but let’s look at some examples. Use them as inspiration for what you can look for in your platform:

  • Build quality indicators

  • Build success over time

  • Build summary report

The build quality indicators report (Figure 7-12) shows a summary of some important values for your builds. Using these data, you can determine whether you’re close to releasing a build. Some of the other information this report contains includes the following:
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig12_HTML.jpg
Figure 7-12

Build quality indicators report

  • Active bugs: The number of active bugs that exist at the time of the build

  • Code churn: The number of lines of code that have been added, removed, and changed in the check-ins before the build

  • Code coverage: The percentage of code covered by tests

  • Inconclusive tests: The number of tests that didn’t succeed or were paused. If the build didn’t succeed, the tests are either not counted or counted as inconclusive.

  • Failed tests: The number of tests that failed during the build

  • Passed tests: The number of tests that passed during the build

The build success over time report (Figure 7-13) shows you the status of the last build for each build category (a combination of build definition, platform, and configuration) run each day. You can use this report to keep track of the quality of the code you check in. Furthermore, for any day on which a build ran, you can view the build summary for that specific day.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig13_HTML.jpg
Figure 7-13

Build success over time report

The build summary report (Figure 7-14) shows you information about test results, test coverage, and code churn, as well as quality notes for each build.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig14_HTML.jpg
Figure 7-14

Build summary report

These metrics are suggestions that you can use as a base for following up on progress and quality in your projects. Different DevOps tools offer different possibilities for reporting and collecting information. Thus, it’s important that you think through what you want for your organization when choosing an DevOps platform.

Using Charts to Monitor Metrics

In TFS and Azure DevOps, you can also add charts as a way of displaying information about your projects. Figure 7-15 shows how charts can be created if you navigate to Queries and then select Charts.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig15_HTML.jpg
Figure 7-15

Accessing Charts editing

You can create charts from many of the queries you create using the query editor. We can choose to create many types of charts (Figure 7-16) including pie, bar, column, stacked bar, and so on, so you can display your results as you want. If you created a chart useful to the entire team, you can add that chart to your dashboard so that it shows up when you open the dashboard. For this functionality to work, you must create your original query as a shared query.
../images/477063_1_En_7_Chapter/477063_1_En_7_Fig16_HTML.jpg
Figure 7-16

Chart information can be displayed in various ways

With Azure DevOps, you can add charts for work item queries to the dashboard from the widget catalog. These charts are configurable. You can choose a query, a chart type, and visualization options while staying in the dashboard context. Chart types include pie, bar, column, stacked bar, line, area, and stacked area. You can also display data in a pivot table. And, you can still add charts from the Charts tab in the work hub and configure them just like you’ve always done.

Summary

Metrics and KPIs are valuable for any organization if it wants to evaluate progress and quality. This chapter showed some examples of metrics used with Agile projects, but they are not limited to these projects. Metrics help you run your projects more efficiently, with greater application quality as an end result.

Keep in mind that different organizations find different metrics valuable. This chapter gave examples of metrics that are commonly used, but there may be others that are better suited for you.

The next chapter walks you through a scenario in which you implement Agile processes on different levels in a software development project.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.119.157.39