This chapter looks at the heart and soul of projects in TFS or VSTS. You will see how all projects build on a process template or just process, as Microsoft calls them these days. The process defines all aspects of your projects including work items types, workflow states, and so on.
ALM Revisited
Having traceability in the ALM processes is key to the successful delivery and maintenance of your applications and systems. Chapter 1 showed that traceability is one of the three cornerstones in a successful ALM solution.
Traceability of relationships between artifacts: Traceability can be a major cost driver in any enterprise if it’s not done correctly. There must be a way of tracing the requirements all the way to delivered code—through architect models, design models, build scripts, unit tests, test cases, and so on. Practices such as test-driven development and configuration management can help, and these can be automated and supported by TFS.
Automation of high-level processes: There are approval processes to control handoffs between analysis and design. There are other handoffs between build, deployment, testing, and so on. Much of this is done manually in many projects, and ALM stresses the importance of automating these tasks for a more effective and less time-consuming process.
Visibility into the progress of development efforts: Many managers and stakeholders have limited visibility into the progress of development projects. Their visibility often comes from steering group meetings during which the project manager goes over the current situation. Other interest groups such as project members may also have limited visibility of the whole project even though they are part of it. This often occurs because reporting is hard to do and can involve a lot of manual work. Daily status reports can quite simply take too much time and effort to produce, for example, especially when we have information in many repositories.
The next section discusses in more detail how work items in TFS can help you accomplish traceability in your projects and organizations.
Traceability
Unfortunately we have seen companies that have stopped making changes to their systems just because no one ever knew where a change (or bug fix) might have its impact. This is not a situation any organization wants to end up with. Yet it is quite common.
At the Swedish Road Administration some years ago, a new version of our system suddenly made old bug fixes disappear. The operators at the Traffic Management Center found themselves with no working phones because of an upgrade. This had the potential to make an accident worse than it already was, because the operators communicate with the rescue team and the police using phones. Having communications suddenly stop working can be a matter of life or death.
The vendor of that piece of software did not have control over its different software versions and did not have a good testing strategy. If the vendor had used automated tests, for instance, they would have discovered broken tests for the bug fix when the fix itself was not included in the next release. By checking which work items were associated with the failed test(s), the vendor would have been able to see which of these contained the problem. This would have indicated why they created the test in the first place, so they could have more easily fixed the problem. This traceability would greatly improve their code.
And if they had used a good configuration management process, they would also have had the capability to trace all versions where the bug fix needed to be inserted, so they wouldn’t forget to include it in the coming releases.
Work item tracking in TFS can help with traceability so you can avoid such problems. Let’s now see how the work item tracking system implements traceability.
The TFS Work Item Tracking System
Sometimes it seems like we can have tons of Post-its on our monitors and desks—each one containing at least one task we are supposed to do. Often it just isn’t possible to track them with a tool. It could be that some tasks are connected with one project, others with another. We could try writing them all down in an Excel sheet and saving that to our computer. But soon we might find that the spreadsheet is located at our laptop, the customer computer, the desktop, at another customer computer, and so on. And we have no idea which one is the current version. This can be a real problem sometimes when we find we have no clue as to which version we should trust.
The same thing is often visible in projects. Project managers have their to-do lists for a project, and they all have their own way of keeping them updated. Let’s say a PM uses Excel to keep track of the tasks—the status of tasks, whom they are assigned to, and so on. How can the PM keep the team updated with the latest to-do list? If the PM chooses to e-mail it, chances are that some won’t save the new version to disk or will just miss it in the endless stream of e-mails coming into the mailbox. Soon there are various versions floating around, and things are generally a mess.
One way to solve this could be to use a project web site running on Microsoft Office SharePoint Server or some other tool like that. This could help, although you could still be in trouble if people forget to save changes or check in the document after they have updated it.
Another problem may occur if, for example, an Excel sheet is updated by a tester who discovers a bug and changes the status of one entry in the task list to indicate that a developer should look at the task again and solve the bug. How can you alert the developer that the bug exists? You would want this action to take place automatically, right? That would be hard if you used only an Excel sheet. The same thing occurs the other way around. When a developer has fixed a bug, you want the tester to be alerted that the problem has been resolved, so the tester can then check whether the bug can be closed.
What about requirements traceability? If the only place you keep track of the connection between requirements and the code is in a document, how do you know that the document is really updated? Can you trust that information?
Even if you purchase a separate tool to help keep track of tasks, it would still be a separate tool for all categories of team members. There are tools for bug tracking, requirements management, test management, and so on—the list can go on for a while. Chances are that someone will forget to update the tool because it takes too long to open or is too difficult to work in or any other excuse for not doing the update. This could cost the project lots of money and time.
Work Items
In TFS you have a task-tracking system at your service. The core of this system is represented by the tasks themselves, called work items. A work item can be pretty much what you want it to be. It can be a bug, a requirement of some sort, a general to-do item, and so on. Each work item represents an object that is stored in the Team Foundation Server database. All work items have a unique ID that helps keep track of the places it is referenced (see Figure 4-1).
Figure 4-1. Each work item has a unique ID
The ID lets you follow one work item, let’s say a requirement, from its creation to its implementation as a piece of executable software (component). Work item IDs are unique across all work item types in all team projects in a project collection. The work item type determines the work item fields that are available for tracking information, defaults defined for each field, and rules and constraints positioned on these fields and other objects that specify the work item workflow. Every change made to a work item field is stored in the work item log, which maintains a historical record of changes ( https://msdn.microsoft.com/en-us/library/vs/alm/work/track/history-and-auditing ).
You can create and modify work items by using Team Explorer, TFS Web, Office Excel, or Office Project. When creating or modifying individual work items, you can work in the work item form, by using Team Explorer or Team Web Access (see Figure 4-2). You can make bulk updates to many work items at a time by using TFS Web, Office Excel, or Office Project.
Figure 4-2. Creating a work item using Web Access
Work items provide a great way to simplify task management in a project while at the same time enabling traceability. No more confusion as to which version of the task list is the current one. No more manual labor for gathering status reports on work progress that are used only at steering group meetings. Now you have a solution that lets you collaborate more easily with your teams and enables all members and stakeholders to view status reports whenever they want. You can also more easily collaborate with people outside the project group by adding work items via the Web.
TFS is so flexible that it lets you tailor the work items as you want them to be. By installing TFS Power Tools, you get an additional menu option called Process Editor under Tools in Visual Studio (see Figure 4-3), which simplifies editing the work items and the whole process as well. From this tool you can modify the work items in the project so they contain new information. Later in this chapter, you will learn more on how you can change your process template, including the work items. If you make a change to the current project (by modifying a work item for example) this affects all new work items you create, but not the existing ones. You only get the change in your current project as well. All new projects created with the same process template will not include these changes, unless you modify the process template on the TFS server.
Figure 4-3. Modifying a work item using the process editor
The work items can contain information in different fields that define the data to be stored in the work item. This means that each field will have a name and a data type. Data types supported in fields are the primitive data types such as string, integer, and double, as well some complex types such as DateTime, PlainText, HTML, and others. System fields are one example of a field (or more correct, a label for a group of fields) that must be present in every work item type, and they represent the minimal recommended subset of fields that any custom work item template should contain. Having such a common subset allows reusing basic Work Item Query Language (WIQL) queries or reports from predefined templates for your custom templates.
All work items can have different information attached to them. You can have information about whom the work item is assigned to and the status of the work at the moment (for example, a bug could be open, closed, under investigation, resolved, and so on). The State field can be modified (see Figure 4-3) so that each work item type can have its own state mechanism. This is logical because a bug probably goes through different states than a general task goes through, for instance. You can also attach documents to the work item and link one work item to other work items. You can create a hierarchy of work items if you want. Say that you implement a requirement as a work item and this requirement contains many smaller tasks. Then you can have the requirement itself at the top and nest the other requirements below that so you know which work items belong to which requirement.
When a bug is discovered, for instance, you can quickly follow the original requirement by its work item ID and see in which places of the code you might have to make some fixes. You can also see the associated work items so that you can evaluate whether other parts of the code need to be changed as a result of this bug fix.
Because TFS saves information about the work item on the data tier, you can see the history of the work item. You can see who created it, who resolved it, who closed it, and so on. The information in the databases can be used for display on reports, allowing you to tailor these depending on your needs. One report could show the status of all bugs, for instance. Stakeholders can see how many open bugs exist, how many are resolved, and much, much more. It is completely up to you how you choose to use the work items.
For those familiar with and used to working with pivot tables, you can use Excel as well to drill down into the information in the TFS data warehouse. There are people who think it is better to use Excel to directly connect to these tables and who use very detailed information in their reports.
The Work Item Form
The work items are defined in the project template in TFS. The template and thus the work item types are defined in a set of XML files stored on the TFS server. The XML file(s) for your work items define what information the work item will include on its form in TFS (see Figure 4-4).
Figure 4-4. The bug form in Microsoft Scrum
As you can see in Figure 4-4, the bug work item type in Microsoft Scrum includes fields for many aspects of the bug. You can assign the bug to a specific person, set its state (status), set its severity, and much more. You can also add a description of the problem and attach files such as screenshots of the bug. There are other options as well, but we will not cover them here.
The fields on the work item form can have properties set for them. You can let a field be read-only, required, automatically populated, and so on. Because you can also change what information is included on this form by editing the XML, you can make it include the information that you want.
We have heard some customers say that they have had problems using the process templates that Microsoft provides because the information required to fill in the forms is not the information they want to track or record. Instead of changing the work item types, they have tried to adapt to the work items. Don’t make this mistake! If you need other information besides what is included in the templates, or if you need the information in another way, change the template. That’s the whole point of having an open and flexible solution such as TFS. You can adjust the tool to fit your needs. I have, for instance, seen the bug work item that Microsoft uses, and it looks nothing like what is included in any of the templates you get with TFS. Instead Microsoft encourages you to adjust the tool to your needs. This includes adjusting the work items.
Work Item Traceability
Consider this little example of how you can use work items to increase traceability. We start with a requirement in the form of a user story:
“As a manager I want to search expense reports so that I can get an overview of expenses more easily.”
This user story is entered into TFS and TFS assigns an ID to it (Figure 4-5). This ID will follow the work item all through its life.
Figure 4-5. Traceability starts with a work item in TFS
Figure 4-6 shows how you can associate the work item with test cases, tasks, and other work items. This means that you can get traceability from a requirement to test cases, to storyboards, and to other work items.
Figure 4-6. Linking work items to features, test cases, and tasks (other work items) enables you to reach great traceability
You can also do this the other way around. When creating storyboards, test cases, or work items, you can of course link them to a new work item (see Figure 4-7) or to another, existing, work item. This is a huge benefit over keeping this information in our heads or on an Excel spreadsheet.
Figure 4-7. Linking work items to new or existing work items
You can also define check-in rules for your developers, which forces them to associate a check-in/changeset with a work item(s), as can be seen in Figure 4-8. There should not be any need for a check-in unless the code change is associated with a work item. You should never make any code changes unless they are required to solve an issue, and this issue should always be documented as a work item.
Figure 4-8. Linking a work item to a check-in can be required by a check-in policy
A changeset in TFS is a logical container into which TFS bundles everything related to a single check-in operation. A changeset consists of:
Source file and folder revisions (adds, renames, edits, deletes, and moves)
Related work items (bugs, etc.)
System metadata (owner, date/time, etc.)
Check-in notes and comments
By associating a build with a changeset, you can create traceability from the original requirement (user story in this case) to the built executable.
This traceability can help you avoid problems like the ones described in the beginning of the chapter. By using the reporting functionality of TFS, you can quickly see what a work item is associated with and hence know that if you change some part of the code (like with a bug fix), this change affects a specific work item. Knowing this you can see that some test cases will be affected by the change and that you need to run those tests again to see if the change broke anything. You can also get warnings from TFS that a check-in affects certain test cases, as you will see later in the book.
Figure 4-9 shows what traceability can look like. From this figure, you can easily understand the importance of traceability and the help you can have from good traceability implementation. You can also see that this way of working would leave you in a much better spot if you, somewhere down the road of a production system, need to implement changes. You can then for example follow the traceability to see which parts of the code are affected by a change and which test cases might need to be changed.
Figure 4-9. Work item traceability
Work Item Queries
In Team Explorer, you can query the work item databases (see Figure 4-10) by using a new query language Microsoft provides: Work Item Query Language (WIQL). It has a SQL-like construct. Figure 4-3 shows an example of a query returning all active bugs, for instance. From Team Explorer or Web Access, you can create new queries or modify existing ones.
Figure 4-10. Work item queries in Web Access
Depending on the process template you use, the work item queries that are supplied differ quite a bit. Microsoft Scrum has different work item queries than the agile template, for instance. When we have used the the agile template in some of my projects, we have found it necessary to add new work item types because the organization needed these for their ALM process. Queries to get information about these new work item types naturally don’t exist, so we have had to make these queries ourselves. Some of these queries have been built during the projects when the need arose, and many of these have later been included in the process template so they are now part of all new projects.
Conclusions on the Work Item Tracking System
The work item tracking system is one of the core components of TFS. This system enables you to create work items, or units of work, and can be used to enable traceability. You can use the work items included with TFS from the beginning, or you can choose to adjust these to your needs, or even create your own work item types. Each work item instance has a unique ID (as you saw earlier in Figure 4-1) that you can attach to the things you do in TFS. This enables you to follow one work item—let’s say a requirement, for example—from its creation to its implementation as a piece of executable software (component). You can also associate one work item with others and build a hierarchy of work items.
When a bug is discovered, you can quickly follow the original requirement by its work item ID and see in which places of the code you might have to make some fixes. You can also see the associated work items so that you can evaluate whether other parts of the code also need to be changed as a result of this bug fix.
If you implement a requirement as a work item, you can use the work item ID to track this requirement through source code and to the final build of the executable system . By requiring all developers to add one or more work item IDs to the check-in using a check-in policy, you can enable this traceability.
Our suggestion is that you look closely at the work item types supplied by Microsoft. Then you can decide which of those you can use for yourself and which you might adjust to suit your organization’s needs. If none of the ones supplied can be used, you have the capability to create your own work item types. Use this opportunity! Don’t adjust your way of working to the Microsoft templates. Adjust TFS to your needs instead.
The Process in TFS
When you create a new team project in TFS or VSTS, you must choose a process for your project. The process was formerly known as process template but the name has changed. The process is a collection of files that defines the features, rules, behaviors, and work items associated with a specific process. TFS and VSTS have the same processes available but there are some differences between the two.
The process defines the work item tracking system as well as other sub-systems you can access from the web portal for a on-premises TFS or through VSTS.
There are three processes defined that you can use:
Agile
CMMI
Scrum
Let us now take a look at these so you know their features and differences.
Agile, CMMI, and Scrum
When you look at the templates you can see that they do not differ very much. The main difference is seen in the work item types they provide. We will soon look at these in more detail but simplified we can say that Scrum is the most lightweight and CMMI (Capability Maturity Model Integration) offers support for a more formal process where formal change management is important.
I will describe these now so that you can better evaluate which model is right for you and your organization.
Scrum
The Scrum process is of course built upon the Scrum framework. Microsoft has worked with Scrum.org when they developed the Scrum process for TFS/VSTS and this is evident when you look at the terminology used in the process.
Figure 4-11 shows work items that work on three different levels. The Scrum process lets you manage work items at the portfolio level as well as the backlog level (work in sprints) as you can see in Figure 4-11 as well. You can see that you have Epics and Feature work items that will help you handle the portfolio backlog level. You can use TFS/VSTS to implement a program team that manages the overall high-level requirements using these two work item types. For the daily work in sprints, one or more teams can have a subset of the portfolio backlog that is dedicated to that specific team while other teams have their own subsets independent of the other teams. In Chapter 6, you will see how you can use the features of TFS/VSTS to set up such a scenario.
Figure 4-11. The Scrum process and some of its work items
Development teams usually work with work items on the backlog level. Here you find product backlog items (PBIs) and their associated tasks. You can also configure bugs and their tasks to be part of this level (see Figure 4-12). These work items are followed up only on remaining work, just like from the agile world.
Figure 4-12. Working with bugs on the backlog level
It is configurable how you work with bug (Figure 4-13). You can allow bugs to be seen on the boards and backlogs on the same level as requirements or tasks. Or you can choose not to have bugs appear on the boards and backlogs at all. You can change the desired behavior when you want so you are not locked in to the first choice you make.
Figure 4-13. You can change how bugs appear in the settings for backlogs and boards
To manage issues that arise during a project, you can use impediments.
Agile
The agile process has been developed in collaboration with the Agile alliance. Originally it was called Microsoft Solutions Framework (MSF) for agile, but now it is only referred to as the agile process in TFS/VSTS.
This process is aimed at supporting agile teams, even those that use Scrum as their process. It tracks development and test activities separately, just like the Scrum process. It also has three levels of work items as you saw in the Scrum process; see Figure 4-14.
Figure 4-14. The agile process in TFS/VSTS
If you compare Figures 4-11 and 4-14, you can see that they are very similar. Only the names seem different. For instance, impediments for Scrum tracking are called issues in the agile process. Likewise, requirements are called user stories in the agile process and product backlog items in the Scrum process. The difference is more evident if you look at how the requirements are documented (see Figures 4-15 and 4-16).
Figure 4-15. The requirements work item in the Scrum process (called a product backlog item)
Figure 4-16. The requirements work item in the agile process (user story)
Comparing these two figures, you can see that much is similar—like the Description and Acceptance criteria. But if you look at the details, you can find things that differ. The field for estimating a requirement is called Effort in the Scrum process and Story Points in the agile process.
Another difference is how you track your progress. In the agile process tasks support tracking Original Estimate, Remaining Work, and Completed Work and in the Scrum process you only track Remaining Work. In my view only remaining work is important but I do realize some organizations have demand for tracking more than this.
Take some time and study the different work item forms in your own environment so that you choose which method is best for your organization. But keep in mind that, as the next chapter will show, you can always customize the process.
Capability Maturity Model Integration (CMMI)
The last process we will talk about is called CMMI . CMMI is a more formal project methodology and Wikipedia summarizes it like this: ”Capability Maturity Model Integration (CMMI) is a process improvement training and appraisal program and service administered and marketed by Carnegie Mellon University (CMU) and required by many DoD and U.S. Government contracts, especially in software development. CMU claims CMMI can be used to guide process improvement across a project, division, or an entire organization. CMMI defines the following maturity levels for processes: Initial, Managed, and Defined. Currently supported is CMMI Version 1.3. CMMI is registered in the U.S. Patent and Trademark Office by CMU.” See more at https://en.wikipedia.org/wiki/Capability_Maturity_Model_Integration .
The work item types in CMMI are also on three levels like the two former processes you looked at (Figure 4-17). In this process, requirements are in fact called requirements and not user stories or PBIs.
Figure 4-17. The CMMI process
Furthermore you can also see that you have more work item types for tracking your projects. You have work items for change requests, issues, reviews, and risk management. Using this process, you can implement a formal change-management process like the one found in, for instance, Information Technology Infrastructure Library (ITIL).
ITIL, formerly an acronym for Information Technology Infrastructure Library, is a set of practices for IT Service Management (ITSM) that focuses on aligning IT services with the needs of business. In its current form (known as ITIL 2011 edition), ITIL is published as a series of five core volumes, each of which covers a different ITSM lifecycle stage. Although ITIL underpins ISO/IEC 20000 (previously BS15000), the International Service Management Standard for IT service management, there are some differences between the ISO 20000 standard and the ITIL framework.
ITIL describes processes, procedures, tasks, and checklists that are not organization-specific, but can be applied by an organization for establishing integration with the organization’s strategy, delivering value, and maintaining a minimum level of competency. It allows the organization to establish a baseline from which it can plan, implement, and measure. It is used to demonstrate compliance and to measure improvement ( https://en.wikipedia.org/wiki/ITIL ).
Figure 4-18 also shows differences between how requirements are documented in CMMI compared to the two other requirements types you have seen. You can quickly see that it is more details that can be filled out by default. And like the agile process, CMMI also supports tracking original estimate, remaining work, and completed work at the task level.
Figure 4-18. The requirements work item in the CMMI process (requirement)
There are other work item types as well in these processes. These are the same for all processes. We will take a look at them shortly but first a few words on work item states.
Workflow States
Workflow states support tracking the status of work as it moves from a new state to a closed or a done state. In TFS/VSTS a workflow consists of a set of states, the valid transitions between the states, and the reasons for transitioning the work item to the selected state (see Figure 4-19).
Figure 4-19. Workflow states and reasons
Figure 4-19 shows the states for the Scrum process, product backlog item. Table 4-1 shows the differences in states for the three processes. Keep in mind that this is configurable, as the next chapter will show.
Table 4-1. The Work Flow States for the Three Processes
Tracking Area | Scrum | Agile | CMMI |
|---|---|---|---|
Workflow states | New Approved Committed Done Removed | New Active Resolved Closed Removed | Proposed Active Resolved Closed |
When you change the state of a work item to removed, closed, or done, the system behaves like this:
Closed or done: Closed or done work items do not appear on the portfolio backlog and backlog pages. However, they do appear on the sprint backlog pages, Kanban board, and task board. Also, when you change the portfolio backlog view to show backlog items (for example, to view Features to product backlog items), items in the closed and done states will appear.
Removed: Work items in this state do not appear on any backlog or board.
Work items are maintained in a team project as long as the team project is active. Even if you set them to closed, done, or removed, a record is kept in the data store, which means you can use them to create queries or reports.
If you need to permanently delete a work item, you can use the witadmin destroywi command-line tool. This tool is not discussed in this book.
Most work items that appear on backlogs and boards support any-to-any transitions. This means you can update the status of a work item using the Kanban board or the task board by dragging it to its corresponding state column.
You can also change the workflow so that you can have the states, transitions, and reasons that you need in your team or organization. More on this in the next chapter.
Work Item Types For All Processes
There are some work item types that are the same for all three processes. They have three different purposes:
Support Microsoft Test Manager (MTM)
Support Feedback Request
Support My Work and Code Review
Let’s now briefly take a look at them.
Work Items that Support MTM
Testers and test managers will often work with these work item types. The goal of this book is not to discuss testing in general, but we include a brief example here. Many test efforts are structured like this:
A test plan is created for a sprint using MTM or the web client. The test plan contains the high-level view of the testing effort. There is a Test Plan work item type included in all processes that you can use.
One or more test suites (another work item type) are created and included in the test plan. The suites themselves include one or more test cases (work item type). The test cases are used to describe step-by-step how a tester should test the application or functionality you are developing. Test cases are often associated with a requirement so that you can see which tests are covering a specific requirement.
So by using test plans, test suites, and test cases, you can structure your testing efforts in a way that gives you traceability and visibility into your tests (see Figure 4-20). You can generate reports and graphs that show the status of these work item types.
Figure 4-20. Workflow states and reasons
There are two more work item types for testing, as you can see in Figure 4-20:
Shared steps: These are test steps that you can reuse in many test cases. They can help you remove redundant test steps by allowing you to reuse them later.
Shared parameters: When you write a manual test, you often want to specify that the test should be repeated several times with different test data. For example, if your users can add different quantities of a product to a shopping cart, you want to check that a quantity of 200 works just as well as a quantity of 1. To do this, you insert parameters in your test steps. Along with the test steps, you provide a table of parameter values. These parameters can be used to create shared parameters so that you can reuse them in other test cases as well.
When you develop your application you usually want stakeholders or end users to provide feedback on what you have done. Using the feedback functionality of TFS/VSTS, you can ask reviewers to provide videos, screenshots, type-written comments, and ratings. Their feedback is captured into work items that you can review and use to create a bug or suggest a new backlog item. There are two work item types you can use:
Feedback request
Feedback response
The last two work item types we will discuss are aimed at code reviews. Code reviews are a critical part of software development. They help you keep your defect count down as well as give you opportunity to learn from other people’s code. A third benefit is that code reviews also allow teams to communicate changes to the application with their peers.
In TFS, you find two work items that help you with the code reviews:
Code review request: This is a request a developer creates and sends to a peer asking for a review of some part of the code.
Code review response: A response gets created when the code review request goes out and the reviewer can choose to accept or reject the review.
Summary
This chapter has discussed many concepts regarding work items and processes in TFS and VSTS. You have seen how the work item tracking system works and how work items can help you increase both visibility and traceability. To put it simply, work items are the core of TFS/VSTS. Almost everything you do involves work items in one way or the other.
Which work items you have at your disposal is determined by the process you choose for your project. Out of the box you’ll find three processes—Scrum, agile, and CMMI. These are similar in many ways but some things are different.
If a process is not sufficient for your needs, you can adjust it and add or remove things as you want. This is the focus of the next chapter.