Windows Live Hotmail is a Web-based e-mail product offered by Microsoft within the Windows Live suite of services. Hotmail is perhaps one of the most well-known of all e-mail services on the Internet and was one of the first free Web-based e-mail services when it launched in 1996. For many years, Hotmail was provided under the MSN brand of services until being rebuilt in managed code and relaunched under the Windows Live brand in 2007. At present, Hotmail is the most widely used Web-based e-mail product in the world and is delivered to more than 250 million users[5] worldwide in more than 35 languages. The key components of the application include Web-based e-mail, calendar, contact management, and instant messaging features.
Since being acquired by Microsoft in 1997, Hotmail has gone through a complete transition, from a Unix-based application to now a managed code application running on Windows Server technologies. It is large, sophisticated, and divided into multiple components including, but not limited to, storage, mail delivery, and Web front-end subsystems. Once deployed, the size of the Web front-end subsystem alone is more than 250 megabytes (MB). This is an indicator of Hotmail’s size as an application, but it in no way represents its complexity.
Because it is primarily an e-mail application, Hotmail must be extremely scalable, reliable, secure, and safe from various Internet-based abuse tactics like spam, viruses, and phishing. Users have come to expect a high level of quality from personal information management applications such as Hotmail, and, consequently, competition with rival products to offer first-class mail, calendar, and contact management software is fierce. To maintain its position as a global leader in Web-based communication, Hotmail’s engineering processes must ensure timely and repeatable delivery of high-quality, reliable, and scalable application code and infrastructure.
The Windows Live Hotmail engineering team has developed a great deal of experience over the years in the delivery of high-quality, reliable, and feature-rich Web applications to a global audience. As a team, they recognize the depth and complexity of the engineering challenges they face and have spent years perfecting their practices. They adhere to a set of guiding principles that govern how they deliver software to market. These principles include:
Perhaps the most important of all features, the team prioritizes this principle above all others. The culture of the team promotes a high degree of product quality, and the team believes that Hotmail should be a reliable, fast, and secure service for its users. This belief permeates the team’s engineering practices from design, coding, and testing through Web site architecture and infrastructure. While many companies strive for a similar culture, the Hotmail team has been able to achieve this by injecting a quality focus into all aspects of its development processes.
All disciplines within the team (which include program management, development, and test) work together to define, develop, and test the software using iterative development methodologies. The Hotmail team has learned from its experience and from the experience of other teams at Microsoft that iterative development improves the quality of the code and reduces bug counts throughout the development cycle while increasing the efficiency of the team. The team employs several tactics to ensure success with iterative development, including keeping iterations short (six to eight weeks), completing features end-to-end prior to starting new features, working in small and autonomous teams, and communicating information about the state of its projects early and often.
Over the years, the Hotmail team has been most successful with its product quality because the team has come to rely on predictable and repeatable processes. For example, common Web server hardware, deployment methodologies, and manageability tools have helped the team gain efficiencies in managing its Web infrastructure, while planning and development processes have helped the team minimize risk to delivery schedules and improve overall delivery quality.
As with most teams at Microsoft, the Hotmail team is part of a larger organization of product development teams within Windows Live. For efficiency, these teams share common tools and processes like source control, build processes, and bug-tracking databases. Overall engineering processes and terminology are also consistent across teams, which ultimately helps streamline the software delivery process as well as improve overall communication between teams.
Since moving away from a waterfall model several years ago, the Hotmail team has found that the above-mentioned set of principles have had a very positive impact on the quality of the code, and the team’s overall agility. These principles collectively have helped the team to continuously innovate on its services, while still achieving a high level of quality and reliability. However, the team cites its move to iterative development and, in particular, specific practices that move quality upstream as the most impactful improvement in its software development processes. More specifically, the team feels that the practices that have had the most positive effect on the quality of its work during iterative development include:
This practice ensures that the development of features proceeds only as fast as features can be verified and tested. Once a feature is coded and ready to test and has passed the required developer-written unit test, it is handed off for a more thorough test pass prior to being checked into the working branch of the source tree. This process ensures that the feature being developed is of high enough quality to be integrated into the working branch of the source code and is not likely to break a full build of the application.
As development of features progresses and testable units are being integrated into the working branch of the source code, full application builds are automatically run each day. Daily builds are accompanied by a set of automated tests, called build verification tests (BVTs), which help ensure that none of the previous day’s source check-ins introduced any new bugs into the core features of the application. This process helps the team manage the overall quality of the application on a daily basis. Build breaks are treated very seriously, and, in the event of a BVT breaking bug, all work on features ceases until the issue is addressed and the suite of tests pass.
This suite of automated tests ensures that basic functionality of the application is of high quality and that the build can be released for more formal end-to-end testing. Automation ensures that tests are repeatable and can be executed rapidly to determine build quality. Executing and passing these tests daily ensures a consistent quality bar throughout the development cycle. At least once per week, the team releases one of these daily builds to an internal "dogfood" environment to allow internal Microsoft early adopters to use the most recent product build and provide feedback to the team.
"Dogfood" is a Microsoft colloquialism for "eating your own dogfood," or conveying confidence in your own products at an early stage of development. The Hotmail team believes in shipping early and often to its users and, as previously mentioned, provides a weekly prerelease version of the product to a small set of Microsoft employees who are eager to evaluate and test a new version of the product. The team believes that adhering to an iterative rhythm of coding, building, testing, and releasing to dogfood helps the team to keep the end-to-end quality of the product as high as possible throughout the development cycle.
During the development of Windows Live Hotmail, which released in 2007, the team moved to a more iterative software development model as it spent time building the entire application in managed code. By leveraging the practices mentioned above, such as building testable units, running daily builds, and releasing to the dogfood environment early and often, the team found that the quality of its work improved dramatically as compared with what it experienced when using a waterfall model. Particularly, the team noted that its bug counts using an iterative development method grew more slowly and peaked at much lower numbers as compared with the waterfall method, which had often left the team with high bug counts at the end of the development cycle. The slower growth rate and lower overall bug counts allowed the team to resolve bugs faster and maintain control over code changes, which ensured the continued stability of the product. The comparison of the bug counts when using a waterfall development model versus an iterative development model are illustrated in 1-4.
The team attributes its success in releasing a high-quality product to the repetitive processes it established to govern its application quality at early stages of the development life cycle. The Hotmail team has proved through experience that implementing quality-focused practices—such as delivering testable units, running daily builds, and using automated test suites—upstream in the development cycle ensures a greater level of quality in the application as well as a more predictable outcome of the development life cycle. In February 2007, this was validated when Hotmail was awarded PC Magazine’s Editor’s Choice award for Web-based e-mail products. This award is clearly a testament to the team’s focus on delivering high-quality, reliable, and scalable application code and infrastructure.
[5] comScore, Inc., press release, February 5, 2008, http://www.comscore.com/press/release.asp?press=2196.