InfoWorld: Rob Carter named one of six “chief technology officers of the year” (February 2002).
As of this writing, Microsoft plans to release an onslaught of new developer tools. We have all but declared war on the current leaders in this configuration management industry. Microsoft has always made it a top priority to listen to what customers want, and this release of developer tools is just another example of delivering on feedback. Software developers have been asking for better tools from Microsoft for years. The delay in getting these tools developed and released is a result of the lack of resources because the focus at Microsoft has been on .NET and security. But now, the Visual Studio team is laser-beam focused on delivering the best products in this developer tools or SCM category.
The following list of “what we’ve heard from our customers about the software life cycle” is taken from a presentation given by Ajay Sudan, the technical product manager for Visual Studio Team System (VSTS), at VSLive on February 7, 2005:
“Building software today is very difficult.”
“My team is spread out and specialized.”
“Tools today don’t work well together.”
“I need to be able to predict the success of my projects.”
“My organization needs customizable process guidance.”
I hope that the previous chapters in this book have addressed the preceding concerns. The tools that we ship help automate, or automagic, the processes that I have outlined. Some important testing and design tools are available, too, but those are advanced topics that can be added only when a good core build process is in place.
In this chapter, I want to touch on the products that are coming out (or that have just been released) that will be helpful to every software development team. This is not intended to be a comprehensive view of these new products, though. Because this book will be released before the products, there might be some changes that I am not able to catch. To address this, I have included links to each topic that point to a site where data is kept up to date.
There’s one last thing I would like to point out, as I did in Chapter 5, “Build Tools and Technologies,” and it’s that everything mentioned in this chapter is just a tool. Consider tools as a means of making your job easier, not of doing your job for you. Do not look at a tool to be the answer to all your problems. Many groups purchase tools and keep adding them to their process. I call this applying band-aids without even looking at what is causing the bleeding. As you can imagine, eventually the band-aids break or there are so many of them that the whole process is doomed. A better analogy would be that adding tools to solve problems is like patching a boat with a weak structure and exterior shell and then taking it out to the ocean. It’ll eventually sink. Hopefully there will be enough life boats for everyone (hint: Titanic reference again).
Every development team should adopt MSBuild.exe as quickly as possible. This tool will eventually replace every build engine that Microsoft currently has available. In Chapter 5, I discuss in detail (“In Steps the 800-Pound Gorilla!”) the different build tools that Microsoft ships. The current plan is to consolidate all the current build tools (MSBuild, VCBuild, and Devenv) into MSBuild as a part of Orcas (the codename for the release after VS 2005). This is likely to change, but that is the plan for now. Let’s talk about MSBuild a little more.
Christope Nasarre has written a great three-part article about MSBuild that is available on MSDN at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnlong/html/msbuildpart1.asp. You should read all three parts if you would like a detailed overview on the product. The site also provides samples that you can download. The following explanation on the basic functionality of MSBuild is taken from “Part 1: From a Project Author’s Perspective.”
The visible side of MSBuild is the msbuild.exe console application that consumes, or uses, project or solution files to build. This executable is just a thin wrapper that translates its command line and drives the real engine hosted by MSBuildEngine.dll. The execution engine is responsible for synchronously running a project composed of tasks defined in an XML project file. Tasks are listed under targets so that you can organize them the way you want. Tasks can take input from properties or items and generate their own output items. Properties are key/value pairs that correspond to the following:
Environment variables
Definitions in the project file
Command-line arguments passed to msbuild.exe using /p
To create a build process, you usually define a set of files to be given to compilers or linkers; therefore, you can define such an array of items in a project file. A task takes input from properties and items, but produces output that can be consumed as input by the next task in the project. You frequently need to set parameter values in a specific way—for example, according to conditions, such as a DEBUG or a RELEASE configuration.
MSBuild consumes project files written in XML that adhere to a schema described by msbuild.xsd. The XML elements correspond to the notions of project, target, task, property, item, and conditions just introduced. This 16KB file is found in the same folder as the command-line tool msbuild.exe: the common language runtime (CLR) folder hosted in the %Sysroot%Microsoft.NETFrameworkv1.2.xxxxxx subdirectory. Microsoft is committed to supporting this format in the long run, and its simplicity provides a valid reason to be confident in it.
This is just a small snippet of the information available at the link at the beginning of this section. Spend a few weeks evaluating the MSBuild. exe tool to see if you can move your builds to it.
Moving along, an ambitious product is coming out of the developer division at Microsoft: Visual Studio Team System (VSTS). I have mentioned it several times in the previous chapters and would like to provide a more involved overview of the product here. Just as a lot of great information on MSBuild is available on the Internet, there is even more on this new tools platform. Following are my two favorite places to look for up-to-date information (you should add these two sites to your favorites list):
http://lab.msdn.microsoft.com/teamsystem/—The homepage for VSTS.http://blogs.msdn.com/somasegar/—Soma’s blog, the VP of the developer division whom I used to work for in the old NT days.
VSTS is a suite of extensible life-cycle tools that helps software teams collaborate to reduce the complexity of delivering modern service-oriented solutions. VSTS expands significantly on Microsoft’s demonstrated successes in delivering highly productive tools by offering businesses tightly integrated and extensible life-cycle tools to increase the predictability of their software development process. With VSTS, organizations can do the following:
Reduce the complexity of delivering modern service-oriented solutions that are designed for operations
Facilitate collaboration among all members of a software team, speeding up development time and ensuring the predictability and reliability of the development process
Customize and extend the team system with their own internal tools and process frameworks, or choose from more than 450 supplemental products from more than 190 partners
VSTS consists of the following:
Visual Studio Team Foundation, which is an extensible team collaboration server that enables all members of the extended IT team to effortlessly manage and track the progress and health of projects
Visual Studio Team Architect, made up of visual designers who enable architects, operations managers, and developers to design service-oriented solutions that can be validated against their operational environments
Visual Studio Team Developer, which is a set of advanced development tools that enable teams to build reliable, mission-critical services and applications
Visual Studio Team Test, which is a set of advanced load-testing tools that enable teams to verify the performance of applications prior to deployment
Visual Studio Team Suite, which is a bundle of Visual Studio Team Architect Edition, Visual Studio Team Developer Edition, and Visual Studio Team Test Edition
MSF, proven process guidance, prescriptive architectural guidance, and solution accelerators that assist organizations in improving the predictability and reliability of delivering mission-critical solutions
I have been to several talks on VSTS, and at each one of them, I see Figure 18.1 or a fancy PowerPoint version of it. It is a great layout of what the product is. The boxes in the figure are self-contained, meaning that you do not need to purchase the other boxes for it to run. I am often asked if VSTS works with other development tools, such as Rational or PVCS. The answer is yes, but you don’t get the optimal effect of the integration that VSTS provides. The source code control tool or bug tracker that you use falls into the Visual Studio Team Foundation (VSTF) box at the bottom of the figure. If you want to use the tools provided by VSTS, you must not mix tools by different manufacturers. If you are interested only in the testing tools of VSTS, it does not matter which VSTF tools you use. Also, when boxes stretch over two or more products, it means those tools are included in the covered products.
This just skims the surface of the development tools that Microsoft is releasing. For the details, I recommend spending a lot of time at the two previously mentioned VSTS sites. Those links have the most current information available.
Because this is a build book, I would like to dive deeper into the details of one of the boxes shown in Figure 18.1—the Build Automation box. This tool is called Visual Studio Team Build (VSTB).
One of the goals of VSTB is to provide a “build lab out of the box” experience. Sure, you can press F5 to get that developer build out, but getting a good public “team build” is a difficult task. That’s because you are building all source code in the team project, including changes by the entire development team, on a separate build machine.
A typical public build involves many steps, including cleaning the remote build machine, getting the sources from the source control on the build machine, compiling, running static analysis, running the build verification or post-build tests, and then communicating the build result to the project team. You might be doing something similar in your build scripts. What VSTB provides is a simple way to kick-start this build process. You go through a simple five-page wizard, and what you generate is a build script.
Figure 18.2 shows a screenshot of the wizard that helps you do this.
This build script automates the entire build process and more. After you have an out-of-the-box build process running out of VSTB, you can edit the build script if you need to further customize or extend the process. To provide this end-to-end integration, VSTB integrates seamlessly with other VSTS tools, such as Team Foundation Source Control, Team Foundation Work Item Tracking, and Team Test.
When the build process is over, a comprehensive report is generated with information about the build result and the general health of the build. Some of the things included in the report are the number of errors and warnings for each build configuration with links to the log files, results of the test runs included in the build, a list of change sets that went into the build, and who checked it in (which could be used to detect the cause of build failure). Other information, such as the code coverage of the tests and work items associated with the build, is listed (which could be used to determine the quality of the build). The report has active links to change sets, work items, and test results for further details.
Let’s examine a scenario to illustrate this a little better. A developer has just looked at a work item assigned to him and fixes his code. While checking into Team Foundation Source Control, he associates the work item with the check-in. VSTB picks up the source for the nightly build, and as a post-build step, it updates the Fixed In field of the work item with the build number. The generated build report lists all the work items that were associated with this build. The tester looks into it to make sure the work item opened by her was resolved in this build and installs the build for further investigation. This is a small example of an integration pointing among the Team Foundation Source Control, Work Item Tracking Tool, and VSTB.
Figure 18.3 explains the VSTB flow.
But what if your organization has some extra steps beyond what VSTB provides out of the box? Fortunately, you can customize VSTB to suit your needs. The underlying build engine is MSBuild, and most of the steps in the build process are MSBuild tasks. All that you need to do is write an MSBuild task that executes the extra steps and include it in the build script that the wizard generates. Through some simple editing, you can specify the order in which this custom step needs to run.
Did you find the information on VSTB interesting? The information is taken from the Team Foundation blog at http://blogs.msdn.com/Team_Foundation. If you go to this site, you will find more incredibly useful information. One thing to note is that VSTB is being designed to work with MSBuild.exe and no other build tool. So before you get excited about adopting it, and unless the specs have changed, you had better roll out MSBuild before spending much time on this tool.
The last new product I want to discuss is Monad, or MSH. The build team lives in the command shell environment, and anyone coming from a UNIX background knows that the Windows command line is weak at best. This tool is not as well known or talked about on the Internet as MSBuild or VSTS. Perhaps the marketing people do not think that many people will care about it. Little do they know, we care!
Monad is Microsoft’s next-generation command-line scripting language. With the advent of a new command-line interface (CLI) (also known as MSH), Microsoft has greatly simplified server administration. Systems administrators can now learn about .NET classes, and developers can easily extend the functionality by adding Monad commandlets (Cmdlets) and providers. This provides a glide-path to higher-level languages, such as C#.
Cmdlets—. Commands are created by Monad from classes that derive from the Cmdlet class and override a well-defined set of methods. Cmdlets define parameters by specifying public properties in their derived class with appropriate attributes. When this is done, Cmdlets can be registered with Monad, which provides both a programmatic and a command-line access to their functionality.
Pipelines—. Pipelines are a sequence of Cmdlets that pass structured data (frequently .NET objects) to the next Cmdlet. This approach provides tremendous leverage and allows the creation of reflection-based Cmdlets (Cmdlets that can operate on incoming data—such as a “where” or a “sort” Cmdlet).
CmdletProvider—. The CmdletProvider namespace defines a set of specific base classes and interfaces. When a developer implements these, the CmdletProvider engine automatically exposes a number of Cmdlets to the user in a common way. It also exposes methods for other Cmdlet developers to access that functionality programmatically.
The Monad Shell has several goals:
Excite and empower—. The Shell excites and empowers Windows system administrators, power users, and developers by delivering a CFCC command-line environment:
A powerful interactive shell and script runtime
A rich procedural scripting language
An endorsed, consistent, and well-documented set of commands and utilities providing comprehensive and fast access to system objects
Innovate—. Leverage unique assets of the Windows platform to deliver key innovations that empower users to accomplish new tasks or perform existing tasks more efficiently.
Secure—. Increase the security of user data for local and remote administration scenarios.
Leverage—. Provide the command-line environment for Longhorn Server Management, .NET Infrastructure Scale-Out (BIG), Advanced Provisioning Framework (APF), Embedded Server Appliance Kit, MOM, WinPE, and so on.
Clean up—. Will make existing/inadequate tools obsolete (for example, cmd.exe1, duplicate utils).
Educate—. Ensure that a user community is created and empowered to be productive and self-supporting with the new capabilities.
Discoverable—. Make certain that users can, through command completion (Intellisense), determine how and what to use to accomplish tasks. Also provide direct means for assisting users to determine which commands to use and how to use them.
Concise—. Create an environment where keystrokes are minimized to maximize efficiency.
I hope you see that the MSH.EXE shell is different from traditional command shells. First, this shell does not use texts as the basis for interaction with the system, but uses an object model based on the .NET platform. This provides a unique way to interact with the system. Second, the list of built-in commands is much longer; this ensures that the interaction with the object model is accomplished with the highest regard to integrity with respect to interacting with the system. Third, the shell provides consistency with regard to interacting with built-in commands through the use of a single parser, rather than relying on each command to create its own parser for parameters.
You will see or have seen several books on the products discussed in this chapter. I wanted only to mention what I thought was relevant to software build teams. Please take all this information with a grain of salt. As of this writing, the information presented here is accurate. Because pressures to release cause features to be dropped, some of what I mention here might not make it to release.
Looking into a crystal ball that predicts the future, you should see:
Adopt MSBuild as soon as possible.
Start researching VSTS and see if there are tools you can implement in your current build process and future processes.
Keep checking back to the links mentioned in this chapter or the
www.thebuildmaster.comWeb site for the most recent information on these products.