The Bigger Picture

NuGet is a free, open source, package management tool for the .NET platform, C++, and JavaScript. It is developed by Microsoft and has been contributed to the ASP.NET Gallery on Outercurve Foundation, formerly known as the CodePlex Foundation. NuGet enables .NET developers to easily find packages, including any dependencies, and manage them within Visual Studio projects or from the command line.

A NuGet package can consist of assemblies, configuration files, source code, and more. Software package management is not a new concept, although the tools have been notably missing for years by .NET developers. Some open source tools were available, but as with many things in life, a chicken-and-egg problem existed between having tooling and having a package publishing ecosystem.

In this chapter, you’ll learn why you need a package management tool, how NuGet compares to other package management tools, and finally, how you can benefit from using NuGet in your projects.

Leveraging the Community

One thing any .NET developer should be aware of is that the .NET ecosystem is much bigger than and not limited to the .NET Framework itself. There are literally thousands of open source libraries and components available for your perusal.

Those who are aware of this vast ecosystem often ask themselves the same questions:

• How do I know if there already is a suitable solution available?
• Where can I find it?
• How do I install it?
• How do I configure it properly?
• How do I know there is a newer version available?

Usually, this meant searching the web for the best solution, trying to find your way through different blog posts, discussion forums, project sites, and hopefully some documentation as well, and eventually pulling some source code or compiled libraries for reference in your projects. The adventure is just beginning! How do you know what to reference? Do I reference all libraries or only some of them? How do I use these libraries?

By the time you were satisfied with your work and got the thing working, you had usually lost valuable time. Of course, the next time you needed this dependency, you already knew what to do—been there, done that! That is, if the library had not evolved into some new major version full of new features and breaking changes, merged assemblies, and obsolete libraries. If you found out about a new, improved version of the dependency during development, justifying an update of this dependency, then this exact same pain could be waiting already!

Wouldn’t it make more sense to leave the hassle of going through all these potential installation and configuration changes up to the people who created it in the first place? Who knows better than the creators of the library about what has changed, or which steps are to be taken to upgrade from an older version to the new one?

If any of this makes sense to you, NuGet will sound like music to your ears. This book will help you leverage NuGet and make sure these situations belong to history.

Get Latest Version

When working on software projects and joining a development team, it’s always pleasant to be able to do a Get Latest Version on the source control system being used, press F5 in Visual Studio, and run the application to get accustomed to it.

Unfortunately, this dream scenario of being able to get latest and run is not always reality. Many times developers reference assemblies from the Global Assembly Cache (GAC) that are not present in a default .NET installation. This means every developer needs to be doing some tooling and library installations on their systems in order to get the application to run.

If you are lucky, smart developers add their external libraries to source control as well, so this get-latest scenario just works. However, there’s still the pain of keeping everything up-to-date.

Did all of this sound awfully familiar? NuGet is the perfect match to overcome such pains in your development process, and this book will help you with optimizing for the get-latest scenario.

The “Not Invented Here” Syndrome

Many development teams, if not most, suffer from the “not invented here” syndrome. While there are several well-tested open source and even commercial solutions available for many problems, teams still tend to build their own. Consider database accessibility: every developer probably has written their own object-relational mapper (ORM) during their career.

But why do we do this? Because that third-party solution is “not invented here.” We tend to build our own solutions and bundle them in a giant, monolithic framework used by the team. Why not package it into smaller components? Why not embrace the third-party ecosystem and have our internal framework depend on it? What happened to the reuse of components?

We’ve all done this, and yet we still don’t seem to be aware of the additional costs associated with reinventing the wheel. Before you know it, you’re in a situation called dependency hell, where you have to maintain, test, and support many versions of different components you wrote, only to support the code base you really care about: the consuming application. This is the code base that creates real value for you, so that’s what you want to focus on.

Escaping from Dependency Hell

To understand the goals of and the reasons behind a package management tool for the .NET platform, it’s worth considering how developers have been managing software dependencies so far. If you’ve ever struggled with updating a referenced assembly to a new version, or searched for documentation on a third-party library to perform a specific task, only to find out such documentation did not exist, you have encountered dependency management issues. If you’ve ever had a build failing or a runtime exception because of a reference collision, you have entered dependency hell.

—Wikipedia, http://en.wikipedia.org/wiki/Dependency_hell

An Introduction to Dependency Management

Following industry best practices, a good software architect, technical lead, or developer designs an application with reusability and modularity in mind. There are tons of good books and resources out there that will explain the reasoning behind this, but that would quickly lead us beyond the scope of this book. Creating or consuming reusable components and libraries is generally considered a good thing. In fact, all developers building on top of the .NET Framework already do this implicitly, without paying too much attention to it, as the .NET Framework is the core framework on top of which we all build our solutions.

Over the years, an increasing number of people in the .NET community initiated open source projects or started contributing to such open source initiatives. Some of them gained a huge amount of popularity due to the quality and maturity they had reached, others due to the unique functionality they provided. Many software projects began referencing these third-party libraries and as such became dependent on them.

Both origins of such software dependencies have one thing in common: you don’t control them. This is exactly the root cause of the frustration that many developers have experienced when fighting dependency issues: you can’t control the third-party reference you rely on.

Imagine you’re part of a development team working on a single module, module X, for an application, as illustrated in Table 1. Your team’s module is depending on two libraries, while one of the other modules has its own dependencies. So far, so good: all teams are working on isolated modules without interdependencies. It seems, however, as shown in Figure 1, that library U, on which module X depends, has a dependency itself to library T from another module. This could be a potential source of headache when one of the two teams requires another version of T, or upgrades T to an incompatible version for library U. You’ve just encountered a version collision on one of your dependencies.

This very simple example illustrates that you don’t need to have a huge dependency graph for an application to encounter dependency issues. Real-life enterprise solutions that can consist of hundreds of modules increase the chance of entering dependency hell exponentially. You can safely conclude that the larger an application’s list of dependencies, and its dependencies’ interdependencies, the more of a pain point this becomes. Because most business applications are more complex than the preceding simple example, most development teams don’t even realize they will hit these issues, until they feel pain and frustration for themselves.

Many development teams have struggled with dependency issues due to incorrect versioning applied to such third-party libraries or components. A typical example would be a library version 1.0 that just released an update using a minor version increment, version 1.1, but nonetheless contains breaking changes. Those companies that hit these issues preferred reinventing the wheel and having control over this dependency. They started creating their own frameworks, trying to solve similar problems. Although this indeed mitigated the risk of dependency issues, it has proven to be a costly workaround due to significantly increased development time, and actually created the illusion of reusability: everyone was applying reusability principles within their own isolated environment. Some smart people realized this, but the reality is that even those people often ended up in an environment where company policies and the economic reality forced them to work this way.

Another approach is to lock down all dependencies at the start of a project. Although you’ll never have this kind of dependency issue, your project will never benefit from any new functionality or improvements that a newer version of those dependencies can bring. A variation to this approach is to unlock the dependencies right after a release, when work starts on the next version. At this point, you pick newer versions of your dependencies, and lock them again until after release. The first approach is harsh, and your project doesn’t benefit from it in the long term, while the latter has been proven quite successful. If you want to be agile and pick any version of a dependency when you feel fit, you’ll be continuously fixing your project instead of driving it toward stable milestones.

You probably have a gut feeling by now that none of these approaches is really satisfying. You’re always running behind the fact you’ll be dealing with all of this yourself, every single time, over and over again. To understand how NuGet solves these issues, it’s important to learn how people came up with answers to these problems over the years, and how they evolved.

A Brief History of Dependency Management

In the early days of the Internet, developers used some sort of repository-based approach, such as the Apache Jakarta Project in Java, where the community surrounded the development platform with lots of open source projects. This helped forge a new approach to dependency management. Around the use of source control, they discouraged checking in binaries and other output of your project. The Apache Ant framework, an XML-based Java build library, helped facilitate this best practice. This tool was the first of its kind because it also allowed you to retrieve files by using HTTP. Typically, an internal site was put in place to be responsible for hosting shared libraries within an organization. At build time, the Ant build files were used to download the libraries locally when not already present. This approach had a shortcoming, however: there was no standard way of setting up these repositories. Also, caching was not uniformly dealt with.

The lessons learned from these experiences were meant to be solved by Java’s second-generation build framework, Maven. Maven was an attempt to standardize the dependency management process and to introduce recommendations around repository management and versioning conventions.

As opposed to what the Unix development community was used to, .NET developers found it a convenient practice to put all referenced dependencies into source control, next to their project—convenient, because it was very easy for getting projects set up. Similar to what the Java community experienced, this approach also lacked a standardized strategy for managing different versions of dependencies.

The release of Microsoft MSBuild was a turning point for the .NET community. This free XML-based build platform is bundled with the .NET Framework and is used by Visual Studio as the underlying build engine. Although a port of Ant was available for .NET at the time, MSBuild quickly became the tool of choice. However, MSBuild did not come with any tasks that supported downloading dependencies.

Inmid-2010, the .NET community seemed to be moving in the right direction with the introduction of OpenWrap, immediately followed by an announcement that Microsoft had joined forces with a .NET package management project called Nubular (Nu). Based on RubyGems, a well-known package management system in the Ruby world, it has been rewritten to get rid of the Ruby dependency. The Nu command-line package management system was now to be called NuPack. Shortly thereafter and following one more rebranding, NuGet was born as the result of a collaborative effort by the Nubular team and Microsoft.

What’s Wrong with What I’ve Always Been Doing?

Before the existence of a package management tool such as NuGet, dependency management in .NET was a big problem. There was no uniform way to tackle dependencies; there was no practical guidance that provided a one-solution-fits-all approach to all development environments.

The various approaches to tackle these problems often relied on the development tools being used. For instance, teams using Team Foundation Server’s source control system could use shared team projects to manage their internal dependencies. On a smaller scale, you could also have Visual Studio projects shared between multiple solutions. Although in some cases this worked, the main issue was that the code in those shared projects could be easily changed in any of those solutions. Such change in the code often resulted in one solution building correctly, while the other solutions were broken all at once. If some kind of version or change management was applied to those solutions, you’d often experience problems with this as well because every single time a solution built the shared project, the other solutions would get a new version as well.

Some organizations built a huge repository of dependencies on a shared network location. Although it looks like a good idea to centralize your dependencies in one giant catalog, this approach really has some huge flaws. For one, referencing dependencies from a network location slows build time significantly, thus reducing efficiency of all development teams. Second, things get even worse if no proper versioning strategy is being used, allowing builds to be overwritten and immediately impacting all development projects depending on the overwritten component.

Others preferred putting all compiled dependencies in a local folder relative to the Visual Studio solution—often called References, Referenced Assemblies, Libs, or something else, clearly an indication that there was no uniform convention. A positive note in this approach is that a proper release has to be picked for every single dependency, including internal ones, which made some sort of change management possible. Until today, this was the most common approach taken by .NET development teams, independent of which software configuration management tool they use. Nevertheless, for those using Team Foundation Server, upgrading a dependency is often a pain in the behind because of the insufficient tracking of binary differences in source control. When a developer changed a DLL in the folder and wanted it to be checked in, he would often be presented with a dialog box indicating that “Team Foundation Server did not detect any changes. Undoing check-out.” As a developer wanting to upgrade an assembly under source control, you had to remember that you needed to explicitly check out the file first, before replacing it and checking it in. This is often used as an argument against Team Foundation Server’s source control system, but, although it is indeed a weakness, the argument is actually an indication that the approach itself is maybe not as perfect as one would think it is.

In general, if you think about it, does it make a lot of sense to do the following:

• Store compiled code in a source control system?
• Duplicate those compiled dependencies relative to a solution all over your source control system?
• Store third-party libraries and components, which may be restricted by license and are the intellectual property of others, in your source control system?

We’ll provide in-depth answers to these questions , where we will guide you through various scenarios and practices to stay as far away as possible from dependency hell.

An Introduction to Package Management

The idea of a package management system for software libraries and components is actually derived from the concept of operating system package managers that install systemwide software packages or libraries, much like apt-get and similar package management tools do in the non-Microsoft space. Linux-based operating systems and other Unix-like systems typically consist of hundreds of distinct software packages, making a package management system a real necessity. On Microsoft Windows, where the .NET runtime relies on dynamic library linking, the runtime also looks in the GAC for shared libraries to remove the issue of DLL hell. As the central repository for shared DLLs, the GAC is often the place into which applications will find or install common assemblies.

In the area of software development, a package manager is responsible for the automation of the installation, update, configuration, and removal process of software packages in a consistent manner within the scope of the solution. Typically, this tool is also responsible for interpreting package metadata such as version information or package dependencies to prevent conflicts or missing prerequisites.

You might be wondering why we discuss package management instead of dependency management. That is because there is a difference between a package and a dependency. A package typically consists of software libraries that, when installed in a target project, form a dependency the target project relies on. But there’s more! A package can also contain source code, configuration files, content such as images, scripts, and other files. Discussing dependencies would limit us to only a part of what a package can contain or what a package manager can contribute to the development process. Package contents and how to create them will be discussed in detail in Chapter 5.

It is also very tempting to confuse a package manager with an installer, often caused by common or very similar terminology and functionality. An installer is usually specific to a given product. In other words, a given product often comes with its own installer. This results in the installer often being tightly coupled to the product being installed, due to product-specific options or installation steps, making it harder or even impossible to reuse the installer. There are also multiple vendors of installers, resulting in different installer formats being used. This is very different from a package manager, which is the one and only installer of packages. Packages need to be of a specific format understood by the package manager. This difference makes a package manager very distinct from an installer.

Because a package manager is aware of package versioning and its dependencies, it is smart enough to detect any potential conflicts. It can add the appropriate binding redirects if needed or warn you upon installation time that a versioning conflict occurred.

This makes installing packages and dependency conflict management more convenient, especially if it can detect and solve issues with binding redirects, or abort package installation by telling you where the problem occurs. Because the format of packages is restricted by a manifest specification (more on that in Chapter 5) and the package manager is the same for all packages, there is a uniform approach independent of the development environment you are working on.

There are only two things you depend on: the package manager and the package publisher. The package manager is a single point of failure that you can be certain has been well tested and functions as it should. Don’t expect major issues from that side. The package publisher, however, is an unknown third-party that is subject to human error. This is not any different from relying on a third-party component without a package manager; you assume that the component or the package contents are well tested and function as expected. Have you seen developers writing unit tests or other kinds of tests on a third-party component before using it? Would it be easy to find the budget to do this? Do you feel this is a natural precaution? The structure of the package content, however, is well-defined according to a manifest that the package manager understands. The only human error the package publisher can make in order to provide you with a dependency issue is an incorrect versioning pattern. Versioning is also a topic of interest in Chapter 5.

Key Benefits of NuGet

NuGet brings you all the benefits of any package manager: it takes care of dependency chains and versioning conflicts at installation time, and it facilitates finding, installing, updating, and uninstalling packages at the application level. Besides these common must-have functionalities, NuGet has much more to offer.

NuGet is more than just a package manager. When changing perspectives, which we will assist you with in Chapter 10, you’ll notice that you can leverage NuGet for other means due to its extensibility, its tooling, its integration, and the fact that it is open source.

NuGet Is Open Source

Although NuGet has been developed by Microsoft, it has been made open source and contributed to the ASP.NET Gallery on Outercurve Foundation. Because it is open source, the .NET community has picked it up quite fast and forged an ecosystem around it by providing tools, guidance, and support. Some even came up with innovative solutions for other scenarios based on NuGet, such as scaffolding, hosted private NuGet feeds in the cloud, systemwide package management, and an application plug-in system. Of course, you can also just get the sources and tweak it further to your needs.

Tooling and Integration

NuGet provides a lot of tooling, allowing us to use NuGet in any .NET environment. Whether you use Visual Studio or another IDE, whether you use Team Foundation Server or another source control system, whether you use the public NuGet Gallery or a custom NuGet feed hosted in the cloud or within company firewalls, you’ll be able to integrate NuGet into your development environment with a minimum of effort. As NuGet becomes a major citizen in our .NET development stack, we feel certain that this integration will become even smoother in the future, as some third-party development tool vendors already demonstrated recently.

The command-line tool allows you to interact with NuGet from a simple console window. This tool can also be leveraged by continuous integration systems or deployment environments to fetch or publish NuGet packages by using some simple commands.

The Package Manager Console, which is part of the NuGet Visual Studio extension, allows you to interact with NuGet from within Visual Studio by using PowerShell. Providing you with useful cmdlets (commandlets) for standard NuGet operations, you’ll be able to manage the NuGet packages your projects rely on without leaving the IDE. The ability to register your own cmdlets via this shell provides you with even greater flexibility.

The NuGet Package Manager Visual Studio extension integrates flawlessly with the Visual Studio IDE, which makes installing a package as easy as adding a reference to a project. The Visual Studio extension also comes with configurable settings for caching and package sources.

NuGet Package Explorer is a graphical user interface allowing you to visualize and edit package contents and metadata. It also allows you to create and publish your packages directly into a feed of choice. The NuGet Package Explorer also has a plug-in system, allowing you to extend this application even further to fit your needs.

All these tools will be used in combination, so you’ll be able to mix them as you want, as well as stick to a tool and work with it through the book. Chapter 2 of this book will help you find and install these tools.

Extensibility

Because NuGet is an open source project, it is also very easy to extend it or its usage. There are some nice examples out there that leverage NuGet and extend its purpose or functionality:

• Chocolatey: A systemwide package manager leveraging NuGet, enabling the quick installation of several useful software packages.
• Entity Framework Code-First: Installing this NuGet package extends the Visual Studio IDE with new functionality around working with Entity Framework Code-First. While you’re not forced to use this tooling, it does automate a lot of things by leveraging the NuGet Package Manager Console.
• MyGet: A “freemium” cloud solution providing you with what we call NuGet-as-a-Service (NaaS). MyGet enables you to avoid the hassle of developing your own NuGet server and to quickly get started leveraging NuGet in your software development environment. They even compile and create packages from your source code.
• ProGet: A shrink-wrap NuGet server for use in development teams.
• Octopus Deploy: A release management system leveraging NuGet to package and host deployable units.

Competition

The widespread usage of NuGet also has another effect: it creates extra competition for Microsoft products. Microsoft is not the only package provider on the NuGet Gallery, and its packages are listed among other third-party components and frameworks. For instance, the choice of using Entity Framework, NHibernate, Dapper.NET, or a similar product is now a matter of picking a package and installing it. If people query the NuGet feed for the word database, they’ll find a whole lot more than just Entity Framework.

This means that it will be equally as easy to find and use a non-Microsoft solution for reference in your project, and based on good reviews of those solutions, you might choose the non-Microsoft package. We think this evolution is good and should result in an improved quality of those solutions. The more people who use it, the better the feedback.

A Wild Future

While not there yet, there are a lot of discussions around NuGet going on around the Internet. One we find particularly interesting is the discussion around the future of Visual Studio. Wouldn’t it be nice if Visual Studio was just a thin layer containing IDE functionality but not all the functionality you’re not using? What if functionality could be installed through NuGet? What if the ASP.NET MVC NuGet package also installed the ASP.NET MVC tooling? We believe there is a bright future for NuGet. And while this book is already in its second edition, we think we’re only at the beginning of unlocking NuGet’s full potential.

Adoption

NuGet has been adopted by most open source projects out there. Many vendors of third-party components are using NuGet to distribute their assemblies and sample code. Products such as MyGet, ProGet, and Sonatype Nexus prove that there is value in offering “private” NuGet repositories for large development teams.

There’s no point in throwing numbers at you in this book, as they would be outdated by the time I finished writing this sentence, but we encourage you to check out some NuGet statistics on the official NuGet Gallery: http://nuget.org/stats. Adoption of NuGet is widely spread!

Who Should Use NuGet

If you feel so familiar with the problems sketched out in this chapter that it scares you, you have definitely been waiting for NuGet to put your mind at ease. If you don’t like to go through the manual process of installing third-party frameworks yourself, or experience issues trying to keep up with new releases of those dependencies, then NuGet is your next best friend. Even if you do know your dependencies very well and are very familiar with its installation, configuration, and different versions, you could benefit from using NuGet by gaining precious time you could spend on more-valuable tasks.

In short, the use of a package manager for .NET development is a best practice every developer should consider, and with the proper guidance of this book, you’ll wonder one day how you managed to live without it.