Metrics and ALM Assessment for Developer Practices
Developer practices are really what Visual Studio 2012 is all about. Microsoft offers several reports and metrics we can use for analyzing and assessing our code base. There are also many assessment questions in the online assessment targeting this area. Let us start with the metrics.
Metrics
Using the metrics from TFS we can find KPIs that will help us understand if we are successfully working to improve our code. These are useful from both architecture and design viewpoints as well as from a developer viewpoint. Using them will help us improve how we design our application or system.
There are several important metrics we can get automatically from Visual Studio and TFS to get a good understanding about the quality of the development work, including
- Code coverage
- Code metrics
- Compiler warnings
- Code analysis warnings
Code coverage
Code coverage shows us how much of the code has been covered by our automated unit tests. We get the value as a percentage of the whole code base. The difficulty is often to decide what percentage is enough. Should we always strive for 100%, or is 80% enough? This is something the team has to discuss with the product owner in Scrum or any other similar decision maker in other processes. This value is input for the Definition of Done (DoD).
Code metrics
There are different code metrics we can get:
- Lines of code is an approximate number based on Intermediate Language (IL) code. A high count might indicate that a type or method is doing too much work and should be split up. This might also be a warning that code will be hard to maintain.
- Class coupling measures the coupling to unique classes through parameters, local variables, return types, method calls, generic or template instantiations, base classes, interface implementations, fields defined on external types, and attribute decoration. Low coupling is better to strive for because high coupling indicates a design that is difficult to reuse and maintain because of its many interdependencies on other types.
- Depth of inheritance indicates the number of class definitions that extend to the root of the class hierarchy. The deeper the hierarchy, the more difficult it might be to understand where particular methods and fields are defined and/or redefined.
- Cyclomatic complexity is created by calculating the number of different code paths in the flow of the program and shows the complexity of the code. A high complexity makes the maintainability suffer, and it can also make it hard to get good code coverage.
- Maintainability index is an index value between 0 and 100 that represents the relative ease of maintaining the code. The higher the better. A rating above 19 is good. Below that, maintainability suffers.
Compiler warnings
Errors and warnings should be avoided in a project. Allowing more than zero errors or warnings tends to result in the team accepting a lower quality on the codebase, which over time will cause the code to lose maintainability (commonly known as the broken windows theory [see http://en.wikipedia.org/wiki/Broken_windows_theory]).
Track this metric to make sure the number of errors is zero. This should ideally be enforced by automatic build policies (as described in Chapter 26).
Code analysis warnings
Code Analysis in Visual Studio performs static analysis on code, which will help developers identify potential design, globalization, interoperability, performance, security, and a bunch of other categories of potential problems.
The Code Analysis tool provides warnings that indicate rule violations in managed code libraries. The warnings are organized into rule areas such as design, localization, performance, and security. Each warning signifies a violation of a Code Analysis rule.
Code analysis can be used to enforce company policies on the code developers write. We can extend the ones Microsoft offers by writing our own rule set or we can suppress the ones we do not want. Definitely discuss this with your development team and the product owner, as the warnings will have an impact on the effort required before the Definition of Done is fulfilled.
Standard Reports
The Quality Indicators report (Agile, CMMI templates) (see Figure 19-1) gives us an overview of the code quality metrics over time. Based on automated builds, the report shows how code churn, code coverage, test results, and the bug count vary over time.
Figure 19-1. Quality Indicators report (from http://msdn.microsoft.com/en-us/library/dd380683(v=vs.110).aspx)
Note The reports in this section are mapped to the particular process template it supports. If you find a report you like, it is possible to customize it to work with another template. In Chapter 32, we describe how to customize an existing report to do this among other things.
Custom Reporting
The reporting capabilities in TFS give us access to most of the information we manage in our ALM process. In the previous section, we have seen how standard reports give us metrics for our project at a general level. By customizing, extending, and creating new reports, we can really get the intelligence to know what works well in our projects and what does not.
Note In chapter 32, we will look at the details of reporting in TFS, including how to create custom reports based on the data models described below.
Code Churn Tables
The code churn tables can be used to query for data about code changes made to code under version control and gives us information such as the number of lines changed, along with when and by whom the changes was made.
See also the Work Item Changeset tables in Chapter 9 as well as the Build Details tables in Chapter 29 for more warehouse data useful for analyzing the development practices.
Assessment
In order to help us evaluate an organization’s maturity in different ALM areas Microsoft has developed its ALM assessments (see Chapter 4). Based on the score of the assessment we received a maturity level for a specific area which we can use for evaluating which direction to take our ALM efforts.
So based on the score we can help the organization reaching the maturity level they need for these areas. The following table (Table 19-1) lists questions that can be used as a basis for an ALM Assessment in the developer practices. The online assessment has many questions covering this area but you might want to consider using some of your own as well.
Table 19-1. ALM Assessment questions
| Area | Sample question | Discussion |
|---|---|---|
| Code Writing | Are there standards in place for writing secure code? | This is implemented using TFS and Visual Studio by adding custom code analysis rules (or use existing) |
| Are there standards for writing coding that ensure comments, variable/function names are free from profanity, political, religious statements? | ||
| Is there a well-defined general coding practice for namespaces, function and variable names? | ||
| Code Analysis | Is there good static code analysis? | If not TFS will help the organization with this. |
| Is there good performance testing? | If not TFS will help the organization with this. | |
| Is there good stress testing? | If not TFS will help the organization with this. | |
| Are there standards for code coverage of tests? | TFS will give these statistics for the development team, thus we can use TFS to implement these standards. | |
| Code reuse | Are patterns & practices established for code reuse? | Code reuse is a good thing. Unfortunately so far we have not seen an organization that has succeeded with implementing this in a good way. Ambition is often high but somehow this never seems to work out. |
| Are Frameworks used? | ||
| xs | Are Code Snippets used? | |
| Code reviews | Are effective code reviews carried out? | If not in place you can suggest that the developer use peer programming which gives code reviews during code writing. |
| Is code often checked in that fails unit tests? | Using automated unit tests and gated check-ins will prevent this. | |
| Quality Metrics | Is there a well-defined and thorough check-in process which includes quality checks? | Here we can definitely use TFS to help us with this. We can set different check-in policies that help us improve quality. |
| Are unit failures measured? | TFS will give us reports of this. | |
| Collaborative Development | Is there an effective versioning and branching strategy? | Most of these topics below are covered by TFS version control system. |
| Is there an effective way source can be retrieved for debugging a deployed product? | ||
| Is there an effective backup and disaster-recovery mechanism in place for source code? | ||
| Does the source control system allow for development activity at different geographical sites? | ||
| Does your source control system support atomic check-ins? | ||
| Does your source control system support branching, merging, diffing, labeling? | ||
| Is there a policy governing multiple checkouts? | ||
| Does the source repository structure and permissions allow for parallel development? | ||
| Version Control Repository | Is all code under effective source control? | Most of these topics below are covered by TFS version control system. |
| Is the source repository well structured? | ||
| Is there a consistent labeling policy? | ||
| Is the source control properly secured? | ||
| Are the source control policies well documented? | ||
| Are all of the organization’s intellectual property (source code, documentation etc.) under effective, secure source control? | ||
| Change Management | Are there formal checks-in criteria governing source code changes? | |
| Is there effective auditing of who makes changes to source control? | ||
| Is there effective auditing of why changes are made to source control? | Implement a check-in policy that requires comments to a check-in. We can also implement a policy that requires an association with a work item. | |
| Is there effective tracking of builds to source control versioning? |
Summary
In this chapter, we have seen how we can use TFS to retrieve information for KPI assessment and also how we can see the development status using standard reports from TFS.
We have also shown how many of the assessment questions from the Microsoft online assessment can help us plan for successful implementation of developer practices.
The next part of this book will cover how we can use TFS to test our system or application.