The movement toward more lightweight software development methodologies has been attributed to the general dissatisfaction with heavyweight, restrictive methods like waterfall. Several of these lightweight methods began emerging in the mid-1990s and emphasized concepts such as self-organizing teams, face-to-face communication, lightweight documentation, and frequent releases. Eventually, several of these methodologies’ creators, including Kent Beck, Martin Fowler, and Ward Cunningham, formed the Agile Alliance[1] and unified the principles of agile software development with the Agile Manifesto.[2]
Agile methodologies tend to encourage iterative approaches to software development with small, self-organizing teams that work in short, very collaborative release cycles. Quality is paramount in every phase of agile software development, and many of the key principles—such as pair programming, test-driven development, refactoring, and continuous integration—ensure that defects are found and eradicated early and often in the release cycle, unlike in the more traditional methodologies.
There are many different agile methodologies in use today, including but not limited to Scrum, eXtreme Programming (XP), test-driven development (TDD), refactoring, and continuous integration. Additionally, there are plenty of great resources available for understanding the principles and practices within each method (see the booklist in Appendix A). It is worth examining a few of the more popular methods of agile in this chapter, and illustrating which of the principles and practices of each has long been favored to improve the quality of software in Microsoft engineering culture.
Scrum is perhaps the best known, or at least the best recognized by name, of the agile methods in use today. At its core, Scrum is not truly a methodology, per se, but rather a framework of practices and defined roles for participants in its processes. Scrum, like most agile methods, encourages small, self-organizing teams that work on a well defined set of development tasks during a short release cycle.
Despite providing a valuable framework for managing the software development process, Scrum itself does little to prescribe any specific methods for managing the quality of the software being developed. This is not necessarily problematic, though, because Scrum pushes accountability to individuals within the Scrum team, which promotes the freedom to implement their own quality practices. Other agile methodologies, however, focus less on project frameworks but are much more prescriptive about specific practices for managing code quality.
Each Scrum project cycle is represented by a Sprint, which is a period of time during which development of a set of features is completed. A Sprint begins with a planning period, where Scrum teams commit to a set of features on which they will focus their efforts. The set of features that are selected by the team during this planning period originate from the Product Backlog, which is a prioritized list of all potential features for the software being developed. By the end of the planning period, all features selected from the Product Backlog are entered and tracked in the Sprint Backlog, which represents the details of the team’s specific feature work. Once the Sprint Backlog is defined, the Sprint can begin. Typically, a Sprint will last 30 days, during which time the team meets each day to review work status and help respective team members remain productive. By the end of the Sprint, the set of features previously defined in the Sprint Backlog is completed and usable. This process is illustrated in 1-2.
Note
Scrum is a really interesting framework to read about. In addition to some of the colorful terminology for participant roles, such as "pigs" and "chickens," there is a lot of value in the framework Scrum provides. I recommend reading Agile Project Management with Scrum, by Ken Schwaber (Microsoft Press, 2004) to learn more.
Scrum is perhaps the most widely used form of agile project management practices within the Microsoft engineering teams. The process Scrum provides for managing work is actually quite effective for small, relatively autonomous feature development teams. Most teams have used Scrum in conjunction with our core engineering practices with great effects.
Unlike Scrum, eXtreme Programming (XP) is not a framework of practices that labels participants or processes with interesting titles or terminology associated with contact sports. XP combines well-defined, valuable set practices into a customer-focused methodology for delivering high-quality software in an iterative manner.
XP proponents assert that ever-evolving software requirements are a natural and desirable part of software development. Rather than attempt to define all requirements in advance of development, XP encourages flexibility in adapting to changing requirements throughout the project cycle. The model relies on a very close working relationship between software developers and their business counterparts, or customers, as shown in 1-3. The XP life cycle is a continuous iteration of feature definition, estimation, feature selection, and feature development. With each project iteration, XP teams become more efficient and effective at delivering solid code with maximum customer impact and within a short development cycle.
While the philosophy and practice of XP has been the subject of debate in recent years over such issues as unstable requirements, lack of documentation, and its inability to scale to large teams, several of the practices that XP promotes are quite valuable for managing code quality. Examples of those methods[3] include:
Developer testing. Developers continually write, and often start by writing unit tests, which must pass for development to continue.
Code refactoring. Developers are always restructuring the system without altering behavior to simplify, reduce duplicity, and add additional flexibility.
Pair programming. Developers work in pairs to reduce coding mistakes.
Continuous integration. Features are integrated and built every time a key development task is completed.
Coding standards. Rules are established that emphasize communication through the code, which developers are required to follow.
These practices are not unique to XP, but they are great examples of quality practices that in many cases transcend other agile methodologies. Several of these methods, such as developer testing, code refactoring, and continuous integration, have been used in practice within Microsoft engineering teams for years and are proven to be quite effective. We will explore these practices within Microsoft engineering teams later in this chapter.
In addition to the agile methods already described, other methods such as test-driven development (TDD) have emerged in recent years. TDD suggests that tests express the requirements with which the code must comply. This technique requires developers to write an automated unit test before writing feature code. As feature code evolves, automated tests provide immediate feedback to the developer and confirm, through true or false assertions, whether the code is behaving correctly. Because each feature of the software subsequently possesses a corresponding test or set of tests, higher confidence in the quality of the code is often the result.
The TDD process[4] for developing features includes the following steps:
Write a test Developers are required to first develop a test that maps to the particular feature requirement.
Run all tests and see the new test fail Assuming that a set of tests exist across the code base, the new test should inevitably fail due to the lack of new feature code.
Write code that makes the test pass Feature code is developed to meet the requirement of the new test.
Run all tests and ensure all succeed Once the new feature code has been written, all tests should be executed again to ensure that new tests pass.
Refactor code Any code written to satisfy the previous steps may not be optimized and should subsequently be refactored to remove duplication or other code bloat.
Rinse and repeat Continue until there are no other requirements left to implement.
Other agile methodologies encourage developer-written unit tests, but TDD stands apart in its rigid view of testing before coding. TDD provides an interesting value proposition as a process that ensures high code quality; however, it does not provide an end-to-end framework for managing the software development life cycle. The goal of TDD is simply to be a framework for addressing customer requirements with software through an iterative approach to testing and coding.
Agile Practices in Microsoft engineering Culture
It is no secret that Microsoft is the largest software engineering company in the world. The company does not earn that title by the number of employees alone but rather the incredible number of products that are delivered each year by Microsoft engineers and development teams across the many continents on which the company maintains an engineering presence.
For its size, Microsoft is an incredibly nimble company, and engineering teams are generally very agile in their approach to delivering software products to market. For example, teams in Microsoft’s Online Services Division, such as Windows Live, are known to deliver major feature updates several times per year to worldwide audiences of hundreds of millions. Consider the size and complexity of products like Microsoft Windows or Microsoft Office. These products require very large, well-coordinated, incredibly precise engineering processes that are highly focused on code quality. Despite the size and complexity of these products, the code base, and the engineering teams, Microsoft still manages to deliver new versions of these products on a fairly frequent basis. (Office releases on average once every two years.) How Microsoft teams accomplish this is based on years of continuous evolution and improvement to engineering practices and automated tooling, which in and of themselves could be the subject of an entire book. However, there are a few of the aforementioned agile practices that are worth mentioning.
Note
For a more in-depth example of how Microsoft teams have continued to evolve engineering practices, I recommend reviewing the Windows Engineering team’s blog at http://blogs.msdn.com/e7/.
For several years, perhaps a decade or more, Microsoft engineering teams have incorporated several "agile" practices for improving quality. Particularly, most teams incorporate coding best practices, developer-written unit tests, refactoring, and continuous integration practices into their daily software development regimen. Teams like the Office team build and deploy the working product on a daily basis. Code from multiple development teams is continuously integrated and tested using highly tuned, automated build and test systems. The result is a daily version of the complete product that is of dependable quality. As you might imagine, these processes require quite an engineering investment, but the result is a "turn-the-crank" process that is optimized for repeatability and high-quality output. Microsoft Visual Studio offers a great set of tools that enable large and small development teams alike to both automate their build processes using MSBuild and automate test coverage using Visual Studio Team System testing tools (VSTS). Both provide capabilities of running daily builds and suites of automated tests in unattended environments.