To build a Smart UI application, a developer first constructs a UI, usually by dragging a series of UI widgets onto a canvas,^[12] and then fills in event handler code for each possible button click or other UI event. All application logic resides in these event handlers: logic to accept and validate user input, to perform data access and storage, and to provide feedback by updating the UI. The whole application consists of these event handlers. Essentially, this is what tends to come out by default when you put a novice in front of Visual Studio.

In this design, there's no separation of concerns whatsoever. Everything is fused together, arranged only in terms of the different UI events that may occur. When logic or business rules need to be applied in more than one handler, the code is usually copied and pasted, or certain randomly chosen segments are factored out into static utility classes. For so many obvious reasons, this kind of design pattern is often called an anti-pattern.

Let's not sneer at Smart UI for too long. We've all developed applications like this, and in fact, the design has genuine advantages that make it the best possible choice in certain cases:

You have probably experienced the disadvantages of this design (anti) pattern firsthand. Such applications become exponentially harder to maintain as each new feature is added: there's no particular structure, so you can't possibly remember what each piece of code does; changes may need to be repeated in several places to avoid inconsistencies; and there's obviously no way to set up unit tests. Within one or two person-years, these applications tend to collapse under their own weight.

It's perfectly OK to make a deliberate choice to build a Smart UI application when you feel it's the best trade-off of pros and cons for a your project (in which case, use classic Web Forms, not ASP.NET MVC, because Web Forms has an easier event model), as long as your business recognizes the limited life span of the resulting software.

Given the limitations of Smart UI architecture, there's a widely accepted improvement that yields huge benefits for an application's stability and maintainability.

By identifying the real-world entities, operations, and rules that exist in the industry or subject matter you're targeting (the domain), and by creating a representation of that domain in software (usually an object-oriented representation backed by some kind of persistent storage system, such as a relational database or a document database), you're creating a domain model. What are the benefits of doing this?

In a .NET application, it makes sense to keep a domain model in a separate assembly (i.e., a C# class library project—or several of them) so that you're constantly reminded of the distinction between domain model and application UI. You would have a reference from the UI project to the domain model project, but no reference in the opposite direction, because the domain model shouldn't know or care about the implementation of any UI that relies on it. For example, if you send a badly formed record to the domain model, it should return a data structure of validation errors, but would not attempt to display those errors on the screen in any way (that's the UI's job).

Model-View Architecture

If the only separation in your application is between UI and domain model,^[13] it's called model-view architecture (see Figure 3-1).

Figure 3-1. Model-view architecture for the Web

It's far better organized and more maintainable than Smart UI architecture, but still has two striking weaknesses:

• The model component contains a mass of repetitious data access code that's specific to the vendor of the particular database being used. That will be mixed in among code for the business processes and rules of the true domain model, obscuring both.

• Since both model and UI are tightly coupled to their respective database and GUI platforms, it's very hard to do unit testing on either, or to reuse any of their code with different database or GUI technologies.

Responding in part to these criticisms, three-tier architecture^[14] cuts persistence code out of the domain model and places that in a separate, third component, called the data access layer (DAL) (see Figure 3-2).

Figure 3-2. Three-tier architecture

Often—though not necessarily—the DAL is built according to the repository pattern, in which an object-oriented representation of a data store acts as a façade on top of a database. For example, you might have a class called OrdersRepository, having methods such as GetAllOrders() or DeleteOrder(int orderID). These will use the underlying database to fetch instances of model objects that match stated criteria (or delete them, update them, etc.). If you add in the abstract factory pattern, meaning that the model isn't coupled to any concrete implementation of a data repository, but instead accesses repositories only through .NET interfaces or abstract base classes, then the model becomes totally decoupled from the database technology. That means you can easily set up unit tests for its logic, using fake or mock repositories to simulate different conditions. You'll see this technique at work in the next chapter.

Three-tier is among the most widely adopted architectures for business software today, because it can provide a good separation of concerns without being too complicated, and because it places no constraints on how the UI is implemented, so it's perfectly compatible with a forms-and-controls-style GUI platform such as Windows Forms or ASP.NET Web Forms.

Three-tier architecture is perfectly good for describing the overall design of a software product, but it doesn't address what happens inside the UI layer. That's not very helpful when, as in many projects, the UI component tends to balloon to a vast size, amassing logic like a great rolling snowball. It shouldn't happen, but it does, because it's quicker and easier to attach behaviors directly to an event handler (a la Smart UI) than it is to refactor the domain model. When the UI layer is directly coupled to your GUI platform (Windows Forms, Web Forms), it's almost impossible to set up any automated tests on it, so all that sneaky new code escapes any kind of rigor. Three-tier's failure to enforce discipline in the UI layer means, in the worst case, that you can end up with a Smart UI application with a feeble parody of a domain model stuck on its side.

Recognizing that even after you've factored out a domain model, UI code can still be big and complicated, MVC architecture splits that UI component in two (see Figure 3-3).

Figure 3-3. MVC architecture for the Web

In this architecture, requests are routed to a controller class, which processes user input and works with the domain model to handle the request. While the domain model holds domain logic (i.e., business objects and rules), controllers hold application logic, such as navigation through a multistep process or technical details like authentication. When it's time to produce a visible UI for the user, the controller prepares the data to be displayed (the presentation model, or ViewData in ASP.NET MVC, which for example might be a list of Product objects matching the requested category), selects a view, and leaves it to complete the job. Since controller classes aren't coupled to the UI technology (HTML), they are just pure application logic. You can write unit tests for them if you want to.

Views are simple templates for converting the view model into a finished piece of HTML. They are allowed to contain basic, presentation-only logic, such as the ability to iterate over a list of objects to produce an HTML table row for each object, or the ability to hide or show a section of the page according to a flag on some object in the view model, but nothing more complicated than that. By keeping them simple, you'll truly have the benefit of separating application logic concerns from presentation logic concerns.

Don't worry if this seems obscure at the moment; soon you'll see lots of examples. If you're struggling to understand how a view could be distinct from a controller, as I did when I first tried to learn MVC architecture (does a TextBox go into a view or into a controller?), it may be because you've only used technologies that make the division very hard or impossible, such as Windows Forms or classic ASP.NET Web Forms. The answer to the TextBox conundrum is that you'll no longer think in terms of UI widgets, but in terms of requests and responses, which is more appropriate for a web application.

You've seen the core design of an MVC application, especially as it's commonly used in ASP.NET MVC; but others interpret MVC differently, adding, removing, or changing components according to the scope and subject of their project.

No doubt you've already experienced the process of brainstorming a domain model in your previous projects. Typically, it involves one or more developers, one or more business experts, a whiteboard, and a lot of cookies. After a while, you'll pull together a first-draft model of the business processes you're going to automate. For example, if you were going to implement an online auctions site, you might get started with something like that shown in Figure 3-4.

Figure 3-4. First-draft domain model for an auctions system

This diagram indicates that the model contains a set of members who each hold a set of bids, and each bid is for an item. An item can have multiple bids from different members.

A key benefit of implementing your domain model as a distinct component is the ability to design it according to the language and terminology of your choice. Strive to find and stick to a terminology for its entities, operations, and relationships that makes sense not just to developers, but also to your business (domain) experts. Perhaps you might have chosen the terms users and roles, but in fact your domain experts say agents and clearances. Even when you're modeling concepts that domain experts don't already have words for, come to an agreement about a shared language, otherwise you can't really be sure that you're faithfully modeling the processes and relationships that the domain expert has in mind. But why is this "ubiquitous language" so valuable?

Be ready to refactor your domain model as often as is necessary. DDD experts say that any change to the ubiquitous language is a change to the software. If you let the software model drift out of sync with your current understanding of the business domain, awkwardly translating concepts in the UI layer despite the underlying impedance mismatch, your model component will become a real drain on developer effort. Aside from being a bug magnet, this could mean that some apparently simple feature requests turn out to be incredibly hard to implement, and you won't be able to explain it to your clients.

Take another look at the auctions example diagram (Figure 3-4). As it stands, it doesn't offer much guidance when it comes to implementation with C# and SQL Server. If you load a member into memory, should you also load all their bids, and all the items associated with those bids, and all the other bids for those items, and all the members who have placed all those other bids? When you delete something, how far does that deletion cascade through the object graph? If you want to impose validation rules that involve relationships across objects, where do you put those rules? If instead of using a relational database, you chose to use a document database, which groups of objects would constitute a single document? And this is just a trivial example-how much more complicated will it get in real life?

The DDD way to break down this complexity is to arrange domain entities into groups called aggregates. Figure 3-5 shows how you might do it in the auctions example.

Figure 3-5. Auctions domain model with aggregates

Each aggregate has a root entity that defines the identity of the whole aggregate, and acts as the "boss" of the aggregate for the purposes of validation and persistence. The aggregate is a single unit when it comes to data changes, so choose aggregates that relate logically to real business processes—that is, the sets of objects that tend to change as a group (thereby embedding further insight into your domain model).

Objects outside a particular aggregate may only hold persistent references to the root entity, not to any other object inside that aggregate (in fact, ID values for nonroot entities don't have to be unique outside the scope of their aggregate, and in a document database, they wouldn't even have IDs). This rule reinforces aggregates as atomic units, and ensures that changes inside an aggregate don't cause data corruption elsewhere.

In this example, members and items are both aggregate roots, because they have to be independently accessible, whereas bids are only interesting within the context of an item. Bids are allowed to hold a reference to members, but members can't directly reference bids because that would violate the items aggregate boundary. Keeping relationships unidirectional, as much as possible, leads to considerable simplification of your domain model and may well reflect additional insight into the domain. This might be an unfamiliar thought if you've previously thought of a SQL database schema as being your domain model (given that all relationships in a SQL database are bidirectional), but C# can model a wider range of concepts.

A C# representation of our domain model so far looks like this:

public class Member
{
    public string LoginName { get; set; } // The unique key
    public int ReputationPoints { get; set; }
}

public class Item
{
    public int ItemID { get; private set; } // The unique key

public string Title { get; set; }
    public string Description { get; set; }
    public DateTime AuctionEndDate { get; set; }
    public IList<Bid> Bids { get; private set; }
}

public class Bid
{
    public Member Member { get; private set; }
    public DateTime DatePlaced { get; private set; }
    public decimal BidAmount { get; private set; }
}

Notice that Bid is immutable (to match how we think of bids in the real world), and the other classes' properties are appropriately protected. These classes respect aggregate boundaries in that no references violate the boundary rule.

Sooner or later you'll have to think about getting your domain objects into and out of some kind of persistent storage—usually a relational, object, or document database. Of course, this concern is purely a matter of today's software technology, and isn't part of the business domain you're modeling. Persistence is an independent concern (real architects say orthogonal concern—it sounds much cleverer), so you don't want to mix persistence code with domain model code, either by embedding database access code directly into domain entity methods, or by putting loading or querying code into static methods on those same classes.

The usual way to keep this separation clean is to define repositories. These are nothing more than object-oriented representations of your underlying database store (or file-based store, or data accessed over a web service, or whatever), acting as a façade over the real implementation. When you're working with aggregates, it's normal to define a separate repository for each aggregate, because aggregates are the natural unit for persistence logic. For example, continuing the auctions example, you might start with the following two repositories (note that there's no need for a BidsRepository, because bid instances need only be found by following references from item instances):

public class MembersRepository
{
    public void AddMember(Member member) { /* Implement me */ }
    public Member FetchByLoginName(string loginName) { /* Implement me */ }

public void SubmitChanges() { /* Implement me */ }
}

public class ItemsRepository
{
    public void AddItem(Item item) { /* Implement me */ }
    public Item FetchByID(int itemID) { /* Implement me */ }
    public IList<Item> ListItems(int pageSize,int pageIndex) { /* Implement me */ }
    public void SubmitChanges() { /* Implement me */ }
}

Notice that repositories are concerned only with loading and saving data, and contain as little domain logic as is possible. At this point, you can fill in the code for each repository method using whatever data access strategy you prefer. You might call stored procedures, but in this example, you'll see how to use an ORM tool (LINQ to SQL) to make your job easier.

We're relying on these repositories being able to figure out what changes they need to save when we call SubmitChanges() (by spotting what you've done to its previously returned entities—LINQ to SQL, NHibernate, and Entity Framework all handle this easily), but we could instead pass specific updated entity instances to, say, a SaveMember(member) method if that seems easier for your preferred data access technique.

Finally, you can get a whole slew of extra benefits from your repositories by defining them abstractly (e.g., as a .NET interface) and accessing them through the abstract factory pattern, or with a DI container. That makes it easy to unit test code that depends on persistence: you can supply a fake or mock repository implementation that simulates any domain model state you like. Also, you can easily swap out the repository implementation for a different one if you later choose to use a different database or ORM tool. You'll see DI at work with repositories later in this chapter.

Microsoft introduced LINQ to SQL in 2007 as part of .NET 3.5. It's designed to give you a strongly typed .NET view of your database schema and data, dramatically reducing the amount of code you need to write in common data access scenarios, and freeing you from the burden of creating and maintaining stored procedures for every type of query you need to perform. It is an ORM tool, not as mature and sophisticated as alternatives such as NHibernate, but sometimes easier to use, considering its full support for LINQ and itsinclusion by default in all editions of Visual Studio 2008 and 2010.

Most demonstrations of LINQ to SQL use it as if it were a quick prototyping tool. You can start with an existing database schema and use a Visual Studio editor to drag tables and stored procedures onto a canvas, and the tool will generate corresponding entity classes and methods automatically. You can then use LINQ queries inside your C# code to retrieve instances of those entities from a data context (it converts LINQ queries into SQL at runtime), modify them in C#, and then call SubmitChanges() to write those changes back to the database.

While this is excellent in a Smart UI application, there are limitations in multilayer architectures, and if you start from a database schema rather than an object-oriented domain model, you've already abandoned a clean domain model design.

There are various different ways to use LINQ to SQL. Here are the two main ones:

As you can guess, the second option requires more work to get started, but it wins in the long term. You can keep persistence concerns separate from the domain classes, and you get total control over how they are structured and how their properties are encapsulated. Plus, you can freely update either the object-oriented or relational representation and update your mapping configuration to match.

The code-first approach isn't too difficult when you get going. Next, you'll see how to build the auctions example domain model and repositories in this way.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Data.Linq.Mapping;
using System.Data.Linq;

[Table(Name="Members")] public class Member
{
    [Column(IsPrimaryKey=true, IsDbGenerated=true, AutoSync=AutoSync.OnInsert)]
    internal int MemberID { get; set; }

    [Column] public string LoginName { get; set; }
    [Column] public int ReputationPoints { get; set; }
}

[Table(Name = "Items")] public class Item
{
    [Column(IsPrimaryKey=true, IsDbGenerated=true, AutoSync=AutoSync.OnInsert)]
    public int ItemID { get; internal set; }

    [Column] public string Title { get; set; }
    [Column] public string Description { get; set; }
    [Column] public DateTime AuctionEndDate { get; set; }

    [Association(OtherKey = "ItemID")]
    private EntitySet<Bid> _bids = new EntitySet<Bid>();
    public IList<Bid> Bids { get { return _bids.ToList().AsReadOnly(); } }
}

[Table(Name = "Bids")] public class Bid
{
    [Column(IsPrimaryKey=true, IsDbGenerated=true, AutoSync=AutoSync.OnInsert)]
    internal int BidID { get; set; }

[Column] internal int ItemID { get; set; }
    [Column] public DateTime DatePlaced { get; internal set; }
    [Column] public decimal BidAmount { get; internal set; }
    [Column] internal int MemberID { get; set; }

    internal EntityRef<Member> _member;
    [Association(ThisKey = "MemberID", Storage = "_member")]
    public Member Member {
        get { return _member.Entity; }
        internal set { _member.Entity = value; MemberID = value.MemberID; }
    }
}

DataContext dc = new DataContext(connectionString); // Get a live DataContext
dc.GetTable<Member>(); // Tells dc it's responsible for persisting the class Member
dc.GetTable<Item>();   // Tells dc it's responsible for persisting the class Item
dc.GetTable<Bid>();    // Tells dc it's responsible for persisting the class Bid
dc.CreateDatabase();   // Causes dc to issue CREATE TABLE commands for each class

DataContext dc = new DataContext(connectionString);
dc.GetTable<Member>().InsertOnSubmit(new Member
{
    LoginName = "Steve",
    ReputationPoints = 0
});
dc.SubmitChanges();

DataContext dc = new DataContext(connectionString);
var members = from m in dc.GetTable<Member>()
              orderby m.ReputationPoints descending
              select m;
foreach (Member m in members)
    Console.WriteLine("Name: {0}, Points: {1}", m.LoginName, m.ReputationPoints);

using System.Data.Linq;
using System.Linq;

public class MembersRepository
{
    private Table<Member> membersTable;
    public MembersRepository(string connectionString) {
        membersTable = new DataContext(connectionString).GetTable<Member>();
    }

    public void AddMember(Member member) {
        membersTable.InsertOnSubmit(member);
    }

    public void SubmitChanges() {
        membersTable.Context.SubmitChanges();
    }

    public Member FetchByLoginName(string loginName)  {
        // If this syntax is unfamiliar to you, check out the explanation
        // of lambda methods near the end of this chapter
        return membersTable.FirstOrDefault(m => m.LoginName == loginName);
    }
}

public class ItemsRepository
{
    private Table<Item> itemsTable;
    public ItemsRepository(string connectionString) {
        DataContext dc = new DataContext(connectionString);
        itemsTable = dc.GetTable<Item>();
    }

    public IList<Item> ListItems(int pageSize, int pageIndex) {
        return itemsTable.Skip(pageSize * pageIndex)
                         .Take(pageSize).ToList();
    }

    public void SubmitChanges() {
        itemsTable.Context.SubmitChanges();
    }

    public void AddItem(Item item) {
        itemsTable.InsertOnSubmit(item);
    }

    public Item FetchByID(int itemID) {
        return itemsTable.FirstOrDefault(i => i.ItemID == itemID);
    }
}

ItemsRepository itemsRep = new ItemsRepository(connectionString);
itemsRep.AddItem(new Item {
    Title = "Private Jet",
    AuctionEndDate = new DateTime(2012, 1, 1),
    Description = "Your chance to own a private jet."
});
itemsRep.SubmitChanges();

A component-oriented design isn't mutually exclusive with a layered design (you can have a general sense of layering in your component graph if it helps), and not everything has to expose an abstract interface—for example, your UI probably doesn't need to, because nothing will depend upon it. Similarly, in a small ASP.NET MVC application, you might choose not to completely decouple your controllers from your domain model—it depends on whether there's enough logic in the domain model to warrant maintaining all the interfaces. However, you'll almost certainly benefit by encapsulating data access code and services inside abstract components.

Be flexible; do what works best in each case. The real value is in understanding the mindset: unlike in a pure layered design where each layer tends to be tightly coupled to the one and only concrete implementation of each lower layer, componentization promotes encapsulation and design-by-contract on a piece-by-piece basis, which leads to greater simplicity and testability.

Component-oriented design goes hand in hand with DI.^[17] DI is a software design pattern that helps you decouple your application components from one another. If you've never used DI before, then you might at first wonder why it's worth bothering with; it may seem like an unnecessary hassle. But trust me—it's worth it! Once you've got it set up, it will make your work simpler, not harder, and you'll get great satisfaction from being able to interchange application components with ease. Let's first talk through some examples.

Imagine you have a class, PasswordResetHelper, that needs to send e-mail and write to a log file. Without DI, you could allow it to construct concrete instances of MyEmailSender and MyLogWriter, and use them directly to complete its task. But then you've got hard-coded dependencies from PasswordResetHelper to the other two components, leaking and weaving their specific concerns and API designs throughout PasswordResetHelper. You can't then design and unit test PasswordResetHelper in isolation; and of course, switching to a different e-mail-sending or log-writing technology will involve considerable changes to PasswordResetHelper. The three classes are fused together. That's the starting point for the dreaded spaghetti code disease.

Avoid this by applying the DI pattern. Create some interfaces that describe arbitrary e-mail-sending and log-writing components (e.g., called IEmailSender and ILogWriter), and then make PasswordResetHelper dependent only on those interfaces:

public class PasswordResetHelper
{
    private IEmailSender _emailSender;
    private ILogWriter _logWriter;

    // Constructor
    public PasswordResetHelper(IEmailSender emailSender, ILogWriter logWriter)
    {
        // This is the DI bit. The constructor demands instances
        // of IEmailSender and ILogWriter, which we save and will use later
        this._emailSender = emailSender;
        this._logWriter = logWriter;
    }

    // Rest of code uses _emailSender and _logWriter
}

Now, PasswordResetHelper needs no knowledge of any specific concrete e-mail sender or log writer. It knows and cares only about the interfaces, which could equally well describe any e-mail-sending or log-writing technology, without getting bogged down in the concerns of any specific one. You can easily switch to a different concrete implementation (e.g., for a different technology), or support multiple ones concurrently, without changing PasswordResetHelper. At runtime, its dependencies are injected into it from outside. And in unit tests, as you'll see later, you can supply mock implementations that allow for simple tests, or ones that simulate particular external circumstances (e.g., error conditions). You have achieved loose coupling.

public interface IMembersRepository
{
    void AddMember(Member member);
    Member FetchByLoginName(string loginName);
    void SubmitChanges();
}

public class AdminController : Controller
{
    IMembersRepository membersRepository;

    // Constructor
    public AdminController(IMembersRepository membersRepository)
    {
        this.membersRepository = membersRepository;
    }

    public ActionResult ChangeLoginName(string oldLogin, string newLogin)
    {
        Member member = membersRepository.FetchByLoginName(oldLogin);
        member.LoginName = newLogin;
        membersRepository.SubmitChanges();

        // ... now render some view
    }
}

A DI container (also called an IoC container) is a standard software component that supports and simplifies DI. It lets you register a set of components (i.e., abstract types and your currently chosen concrete implementations), and then handles the business of instantiating them. You can configure and register components either with C# code or an XML file (or both).

At runtime, you can call a method similar to container.Resolve(Type type), where type could be a particular interface or abstract type, or a particular concrete type, and the container will return an object satisfying that type definition, according to whatever concrete type is configured. It sounds trivial, but a good DI container adds three clever features:

So, in the preceding example, you'd configure MembersRepository as the active concrete implementation for IMembersRepository. Then, when some code calls container.Resolve(typeof(AdminController)), the container will figure out that to satisfy AdminController's constructor parameters it first needs an object implementing IMembersRepository. It will get one according to whatever concrete implementation you've configured (in this case, MembersRepository), supplying the connectionString you've configured. It will then use that to instantiate and return an AdminController.

protected void Application_Start()
{
    RegisterRoutes(RouteTable.Routes);
    ControllerBuilder.Current.SetControllerFactory(new NinjectControllerFactory());
}

In the .NET world, you can choose from a range of open source and commercial unit test frameworks, the most widely known of which is NUnit. Typically, you create a separate class library project in your solution to hold test fixtures (unless Visual Studio has already created one for you). A test fixture is a C# class that defines a set of test methods—one test method per behavior that you want to verify.

Here's an example test fixture, written using NUnit, that tests the behavior of AdminController's ChangeLoginName() method from the previous example:

[TestFixture]
public class AdminControllerTests
{
    [Test]
    public void Can_Change_Login_Name()
    {
        // Arrange (set up a scenario)
        Member bob = new Member { LoginName = "Bob" };
        FakeMembersRepository repos = new FakeMembersRepository();
        repos.Members.Add(bob);
        AdminController controller = new AdminController(repos);

        // Act (attempt the operation)
        controller.ChangeLoginName("Bob", "Anastasia");

        // Assert (verify the result)
        Assert.AreEqual("Anastasia", bob.LoginName);
        Assert.IsTrue(repos.DidSubmitChanges);
    }

    private class FakeMembersRepository : IMembersRepository
    {
        public List<Member> Members = new List<Member>();
        public bool DidSubmitChanges = false;

        public void AddMember(Member member) {
            throw new NotImplementedException();
        }

public Member FetchByLoginName(string loginName){
            return Members.First(m => m.LoginName == loginName);
        }

        public void SubmitChanges(){
            DidSubmitChanges = true;
        }
    }
}

This test fixture uses a test-specific fake implementation of IMembersRepository to simulate a particular condition (i.e., there's one member in the repository: Bob). Next, it calls the method being tested (ChangeLoginName()), and finally verifies the result using a series of Assert() calls. You can run your tests using one of many freely available test runner GUIs,^[19] such as NUnit GUI (see Figure 3-8).

Figure 3-8. NUnit GUI showing a green light

NUnit GUI finds all the [TestFixture] classes in an assembly, and all their [Test] methods, letting you run them either individually or all in sequence. If all the Assert() calls pass and no unexpected exceptions are thrown, you'll get a green light. Otherwise, you'll get a red light and a list of which assertions failed.

It might seem like a lot of code to verify a simple behavior, but it wouldn't be much more code even if you were testing a very complex behavior. As you'll see in later examples in this book, you can write far more concise tests, entirely eliminating fake test classes such as FakeMembersRepository, by using a mocking tool.

TDD and the Red-Green-Refactor Workflow

You're off to a good start with unit testing. But how can your unit tests help you design your code unless you write the tests before the code itself? And how do you know whether your tests actually prove something? What if you accidentally missed a vital Assert(), or didn't set up your simulated conditions quite right, so that the test gives a false positive?

TDD prescribes a development workflow called red-green-refactor, an approach to writing code that implicitly tests your tests. The basic workflow is as follows:

1. Decide that you need to add a new behavior to your code. Write a unit test for the behavior, even though you haven't implemented it yet.

2. See the test fail (red).

3. Implement the behavior.

4. See the test pass (green).

5. If you think the code could be improved by being restructured—for example, by reorganizing or renaming methods or variables but without changing the behavior, do that now (refactor). Afterward, the tests should still pass.

6. Repeat.

The fact that the test result switches from red to green, even though you don't change the test, proves that it responds to the behavior you've added in the code.

Let's see an example. Earlier in this chapter, during the auctions example, there was planned to be a method on Item called AddBid(), but we haven't implemented it yet. Let's say the behavior we want is,"You can add bids to an item, but any new bid must be higher than all previous bids for that item." First, add a method stub to the Item class:

public void AddBid(Member fromMember, decimal bidAmount)

{
    throw new NotImplementedException();
}

Note

You don't have to write method stubs before you write test code. You could just write a unit test that tries to call AddBid() even though no such method exists yet. Obviously, there'd be a compiler error. You could think of that as the first failed test.Or, if you prefer to skip that ceremony, you can just add method stubs as you're going along.

It may be obvious that this code doesn't have the desired behavior, but that doesn't stop you from writing a unit test:

[TestFixture]
public class AuctionItemTests
{
    [Test]
    public void Can_Add_Bid()
    {
        // Set up a scenario
        Member member = new Member();
        Item item = new Item();

        // Attempt the operation
        item.AddBid(member, 150);

        // Verify the result
        Assert.AreEqual(1, item.Bids.Count());
        Assert.AreEqual(150, item.Bids[0].BidAmount);
        Assert.AreSame(member, item.Bids[0].Member);
    }
}

Run this test, and of course you'll get a red light (NotImplementedException). It's time to create a first-draft implementation for Item.AddBid():

public void AddBid(Member fromMember, decimal bidAmount)
{
    _bids.Add(new Bid{
        Member = fromMember,
        BidAmount = bidAmount,
        DatePlaced = DateTime.Now,
        ItemID = this.ItemID
    });
}

Now if you run the test again, you'll get a green light. So this proves you can add bids, but says nothing about new bids being higher than existing ones. Start the red-green cycle again by adding two more tests:

[Test]
public void Can_Add_Higher_Bid()
{
    // Set up a scenario
    Member member1 = new Member();
    Member member2 = new Member();
    Item item = new Item();

    // Attempt the operation
    item.AddBid(member1, 150);
    item.AddBid(member2, 200);

    // Verify the result
    Assert.AreEqual(2, item.Bids.Count());
    Assert.AreEqual(200, item.Bids[1].BidAmount);
    Assert.AreSame(member2, item.Bids[1].Member);
}

[Test]
public void Cannot_Add_Lower_Bid()
{
    // Set up a scenario
    Item item = new Item();
    item.AddBid(new Member(), 150);

    // Attempt the operation
    try
    {
        item.AddBid(new Member(), 100);
        Assert.Fail("Should throw exception when invalid bid attempted");
    }
    catch (InvalidOperationException) { /* Expected */ }
}

Run all three tests together, and you'll see that Can_Add_Bid and Can_Add_Higher_Bid both pass, whereas Cannot_Add_Lower_Bid fails, showing that the test correctly detects a failure to prevent adding lower bids (see Figure 3-9).

Figure 3-9. NUnit GUI shows that we failed to prevent adding lower bids.

Of course, there isn't yet any code to prevent you from adding lower bids. Update Item.AddBid():

public void AddBid(Member fromMember, decimal bidAmount)
{
    if ((Bids.Count() > 0) && (bidAmount <= Bids.Max(b => b.BidAmount)))
        throw new InvalidOperationException("Bid too low");
    else
    {
        _bids.Add(new Bid
        {
            Member = fromMember,
            BidAmount = bidAmount,
            DatePlaced = DateTime.Now,
            ItemID = this.ItemID
        });
    }
}

Run the tests again and all three will pass! And that, in a nutshell, is TDD. We drove the development process by specifying a sequence of required behaviors (first, you can add bids, and second, you can't add lower bids). We represented each specification as a unit test, and the code to satisfy them followed.

For web applications, the most common approach to integration testing is UI automation, which means automating a web browser—simulating an end user clicking links and submitting forms—to exercise the application's entire technology stack. The two best-known open source browser automation options for .NET developers are

Here's an example integration test, written using NUnit and WatiN, for the default application that Visual Studio gives you when you create a brand new ASP.NET MVC 2 web application. It checks that once a user is logged in, their login name appears in the page header area.

[TestFixture]
public class UserAccountTests
{
    private const string rootUrl = "http://localhost:8080";

    [Test]
    public void DisplaysUserNameInPageHeader()
    {

var userName = "steve";
        var password = "mysecret";

        // Register a new account
        using (var browser = CreateBrowser()) {
            browser.GoTo(rootUrl + "/Account/Register");
            browser.TextField("UserName").Value = userName;
            browser.TextField("Email").Value = "[email protected]";
            browser.TextField("Password").Value = password;
            browser.TextField("ConfirmPassword").Value = password;
            browser.Forms[0].Submit();
        }

        // Log in and check the page caption
        using (var browser = CreateBrowser()) {
            browser.GoTo(rootUrl + "/Account/LogOn");
            browser.TextField("UserName").Value = userName;
            browser.TextField("Password").Value = password;
            browser.Forms[0].Submit();
            browser.GoTo(rootUrl);
            string actualHeaderText = browser.Element("logindisplay").Text;
            StringAssert.Contains("Welcome " + userName + "!", actualHeaderText);
        }
    }

    // Just using IE here, but WatiN can automate Firefox too
    private Browser CreateBrowser() { return new IE(); }
}

This integration test has a number of benefits over a unit test for the same behavior:

At the same time, there are drawbacks:

The speed and test data issues are fundamental, but to address the maintenance issue, let's consider a different approach to structuring integration tests.

Scenario: See my account name in the page header
        Given I have registered as a user called "Steve"
          And I am logged in as "Steve"
        When I go to the homepage
        Then the page header should display "Welcome Steve!"

[Binding]
public class Homepage
{
    [When(@"I go to the homepage")]
    public void WhenIGoToTheHomepage() {
        WebBrowser.Current.GoTo(WebBrowser.RootUrl);
    }

    [Then(@"the page header should display ""(.*)""")]
    public void ThenThePageHeaderShouldDisplay(string text) {
        string actualHeaderText = WebBrowser.Current.Element("logindisplay").Text;
        StringAssert.Contains(text, actualHeaderText);
    }
}

[Binding]
public class UserRegistration
{
   private Dictionary<string, string> passwords = new Dictionary<string, string>();

   [Given(@"I have registered as a user called ""(.*)""")]
   public void GivenIHaveRegisteredAsAUserCalled(string userName) {
       passwords[userName] = Guid.NewGuid().ToString();
       WebBrowser.Current.GoTo(WebBrowser.RootUrl + "/Account/Register");
       WebBrowser.Current.TextField("UserName").Value = userName;
       WebBrowser.Current.TextField("Email").Value = "[email protected]";
       WebBrowser.Current.TextField("Password").Value = passwords[userName];
       WebBrowser.Current.TextField("ConfirmPassword").Value = passwords[userName];
       WebBrowser.Current.Forms[0].Submit();
   }

   [Given(@"I am logged in as ""(.*)""")]
   public void GivenIAmLoggedInAs(string userName) {

WebBrowser.Current.GoTo(WebBrowser.RootUrl + "/Account/LogOff");
       WebBrowser.Current.GoTo(WebBrowser.RootUrl + "/Account/LogOn");
       WebBrowser.Current.TextField("UserName").Value = userName;
       WebBrowser.Current.TextField("Password").Value = passwords[userName];
       WebBrowser.Current.Forms[0].Submit();
   }
}

Almost all the new language features in C# 3 have one thing in common: they exist to support language-integrated query (LINQ). The idea of LINQ is to make data querying a native feature of the language, so that when you're selecting, sorting, filtering, or transforming of sets of data—whether it's a set of .NET objects in memory, a set of XML nodes in a file on disk, or a set of rows in a SQL database—you can do so using one standard, IntelliSense-assisted syntax in your C# code (and using far less code).

As a very simple example, in C# 2, if you wanted to find the top three integers in an array, you'd write a function like this:

int[] GetTopThreeValues(int[] values)
{
    Array.Sort(values);
    int[] topThree = new int[3];
    for (int i = 0; i < 3; i++)
        topThree[i] = values[values.Length − i − 1];
    return topThree;
}

whereas using LINQ, you'd simply write this:

var topThree = (from i in values orderby i descending select i).Take(3);

Note that the C# 2 code has the unfortunate side effect of destroying the original sort order of the array—it's slightly trickier if you want to avoid that. The LINQ code does not have this problem.

At first, it's hard to imagine how this strange, SQL-like syntax actually works, especially when you consider that much more complex LINQ queries might join, group, and filter heterogeneous data sources. Let's consider each one of the underlying mechanisms in turn, not just to help you understand LINQ, but also because those mechanisms turn out to be useful programming tools in their own right, and you need to understand their syntax to use ASP.NET MVC effectively.

Have you ever wanted to add an extra method to a class you don't own? Extension methods give you the syntactic convenience of "adding" methods to arbitrary classes, even sealed ones, without letting you access their private members or otherwise compromising on encapsulation.

For example, a string doesn't by default have a method to convert itself to title case (i.e., capitalizing the first letter of each word), so you might traditionally define a static method to do it:

public static string ToTitleCase(string str)
{
    if (str == null)
        return null;
    else
        return CultureInfo.CurrentUICulture.TextInfo.ToTitleCase(str);
}

Now, by placing this static method into a public static class, and by using the this keyword in its parameter list, as in the following code:

public static class MyExtensions
{
    public static string ToTitleCase(this string str)
    {
        if (str == null)
            return null;
        else
            return CultureInfo.CurrentUICulture.TextInfo.ToTitleCase(str);
    }
}

you have created an extension method (i.e., a static method that takes a this parameter). The C# compiler lets you call it as if it were a method on the .NET type corresponding to the this parameter—for example:

string place = "south west australia";
Console.WriteLine(place.ToTitleCase()); // Prints "South West Australia"

Of course, this is fully recognized by Visual Studio's IntelliSense. Note that it doesn't really add an extra method to the string class. It's just a syntactic convenience: the C# compiler actually converts your code into something looking almost exactly like the first nonextension static method in the preceding code, so there's no way you can violate any member protection or encapsulation rules this way.

There's nothing to stop you from defining an extension method on an interface, which creates the previously impossible illusion of having code automatically shared by all implementers of an interface. The following example uses the C# 2 yield return keyword to get all the even values out of an IEnumerable<int>:

public static class MyExtensions
{
    public static IEnumerable<int> WhereEven(this IEnumerable<int> values)
    {
        foreach (int i in values)
            if (i % 2 == 0)
                yield return i;
    }
}

You'll now find that WhereEven() is available on List<int>, Collection<int>, int[], and anything else that implements IEnumerable<int>.

If you wanted to generalize the preceding WhereEven() function into an arbitrary Where<T>() function, performing an arbitrary filter on an arbitrary data type, you could use a delegate, like so:

public static class MyExtensions
{
    public delegate bool Criteria<T>(T value);
    public static IEnumerable<T> Where<T>(this IEnumerable<T> values,
                                                        Criteria<T> criteria)
    {
        foreach (T item in values)
            if (criteria(item))
                yield return item;
    }
}

Now you could, for example, use Where<T> to get all the strings in an array that start with a particular letter, by passing a C# 2 anonymous delegate for its criteria parameter:

string[] names = new string[] { "Bill", "Jane", "Bob", "Frank" };
IEnumerable<string> Bs = names.Where<string>(
                            delegate(string s) { return s.StartsWith("B"); }
                         );

I think you'll agree that this is starting to look quite ugly. That's why C# 3 introduces lambda methods (well, it borrows them from functional programming languages), which have simplified syntax for writing anonymous delegates. The preceding code may be reduced to

string[] names = new string[] { "Bill", "Jane", "Bob", "Frank" };
IEnumerable<string> Bs = names.Where<string>(s => s.StartsWith("B"));

That's much tidier, and even starts to read a bit like an English sentence. In general, lambda methods let you express a delegate with any number of parameters using the following syntax:

(a, b, c) => SomeFunctionOf(a, b, c)

If you're describing a delegate that takes only one parameter, you can drop the first set of brackets:

x => SomeFunctionOf(x)

You can even put more than one line of code into a lambda method, finishing with a return statement:

x => {
         var result = SomeFunctionOf(x);
         return result;
     }

Once again, this is just a compiler feature, so you're able to use lambda methods when calling into a .NET 2.0 assembly that expects a delegate.

Actually, the previous example can be made one step simpler:

string[] names = new string[] { "Bill", "Jane", "Bob", "Frank" };
IEnumerable<string> Bs = names.Where(s => s.StartsWith("B"));

Spot the difference. This time, we haven't specified the generic parameter for Where<T>()—we just wrote Where(). That's another one of the C# 3 compiler's party tricks: it can infer the type of a function's generic argument from the parameters of a delegate (or lambda method) passed to it. (The C# 2 compiler had some generic type inference abilities, but it couldn't do this.)

Now we have a totally general purpose Where() operator with a tidy syntax, which takes you a long way toward understanding how LINQworks.

This may seem like a strange tangent in this discussion, but bear with me. Most of us C# programmers are, by now, quite bored of writing properties like this:

private string _name;
public string Name
{
    get { return _name; }
    set { _name = value; }
}

private int _age;
public int Age
{
    get { return _age; }
    set { _age = value; }
}

// ... and so on

So much code, so little reward. It makes you tempted just to expose a public field on your class, considering that the end result is the same, but that would prevent you from ever adding getter or setter logic in the future without breaking compatibility with assemblies you've already shipped (and screwing up data binding). Fortunately, our hero the C# 3 compiler is back with a new syntax:

public string Name { get; set; }
public int Age { get; set; }

These are known as automatic properties. During compilation, the C# 3 compiler automatically adds a private backing field for each automatic property (with a weird name you'll never access directly), and wires up the obvious getters and setters. So now you have the benefits without the pain. Note that you can't omit the get; or set; clauses to create a read-only or write-only field; you add an access modifier instead—for example:

public string Name { get; private set; }
public int Age { internal get; set; }

Should you need to add custom getter or setter logic in the future, you can convert these to regular properties without breaking compatibility with anything. There's a missing feature, though—there's no way to assign a default value to an automatic property as you can with a field (e.g., private object myObject = new object();), so you have to initialize them during your constructor, if at all.

Here's another common programming task that's quite boring: constructing objects and then assigning values to their properties. For example

Person person = new Person();
person.Name = "Steve";
person.Age = 93;
RegisterPerson(person);

It's one simple task, but it takes four lines of code to implement it. Just when you were on the brink of getting RSI, the C# 3 compiler swoops in with a new syntax:

RegisterPerson(new Person { Name = "Steve", Age = 93 });

So much better! By using the curly brace notation after a new expression, you can assign values to the new object's publicly settable properties, which is great when you're just creating a quick new instance to pass into a method. The code within the curly braces is called an object initializer, and you can put it after a normal set ofconstructor parameters if you need. Or, if you're calling a parameterless constructor, you can simply omit the normal constructor parentheses.

Along similar lines, the C# 3 compiler will generate some code for you if you're initializing a new collection—for example:

List<string> countries = new List<string>();
countries.Add("England");
countries.Add("Ireland");
countries.Add("Scotland");
countries.Add("Wales");

can now be reduced to this:

List<string> countries = new List<string> {
    "England", "Ireland", "Scotland", "Wales"
};

The compiler lets you use this syntax when constructing any type that exposes a method called Add(). There's a corresponding syntax for initializing dictionaries, too:

Dictionary<int, string> zipCodes = new Dictionary<int,string> {
    { 90210, "Beverly Hills" },
    { 73301, "Austin, TX" }
};

C# 3also introduces the var keyword, in which a local variable is defined without specifying an explicit type; the compiler infers the type from the value being assigned to it—for example:

var now = new DateTime(2001, 1, 1); // The variable takes the type DateTime
int dayOfYear = now.DayOfYear;      // This is legal
string test = now.Substring(1, 3);  // Compiler error! No such function on DateTime

This is called type inference or implicit typing. Note that, although many developers misunderstand this point at first, the var keyword does not create dynamically typed variables (e.g., in the sense that all variables are dynamically typed in JavaScript, or in the sense of C#4's dynamic invocation). After compilation, the variable is just as explicitly typed as ever—the only difference is that the compiler works out what type it should be instead of being told. Implicitly typed variables can only be used in a local method scope: you can't use var for a class member or as a return type.

An interesting thing happens at this point. By combining object initializers with type inference, you can construct simple data storage objects without ever having to define a corresponding class anywhere—for example:

var salesData = new { Day = new DateTime(2009, 01, 03), DollarValue = 353000 };
Console.WriteLine("In {0}, we sold {1:c}", salesData.Day, salesData.DollarValue);

Here, salesData is an anonymously typed object. Again, that doesn't mean it's dynamically typed; it's of some real .NET type that you just don't happen to know (or care about) the name of. The C# 3 compiler will generate an invisible class definition on your behalf during compilation. Note that Visual Studio's IntelliSense is fully aware of what's going on here, and will pop up the appropriate property list when you type salesData., even though the type it's prompting you about doesn't even exist yet. Clever stuff indeed.

The compiler generates a different class definition for each combination of property names and types that you use to build anonymously typed objects. So, if two anonymously typed objects have the same property names and types, then at runtime they'll actually be of the same .NET type. This means you can put corresponding anonymously typed objects into an anonymously typed array—for example:

var dailySales = new[] {
    new { Day = new DateTime(2009, 01, 03), DollarValue = 353000 },
    new { Day = new DateTime(2009, 01, 04), DollarValue = 379250 },
    new { Day = new DateTime(2009, 01, 05), DollarValue = 388200 }
};

For this to be allowed, all the anonymously typed objects in the array must have the same combination of property names and types. Notice that dailySales is still introduced with the var keyword, never var[], List<var>, or anything like that. Because var means "whatever fits," it's always sufficient on its own, and retains full type safety both at compile time and runtime.

public delegate TDest Transformation<TSrc, TDest>(TSrc item);

public static class MyExtensions
{
    public static IEnumerable<TDest> Select<T, TDest>(this IEnumerable<T> values,
                                                    Func<T, TDest> transformation)
    {
        foreach (T item in values)
            yield return transformation(item);
    }
}

// Prepare sample data
string[] nameData = new string[] { "Steve", "Jimmy", "Celine", "Arno" };

// Transform onto an enumerable of anonymouslytyped objects
var people = nameData.Where(str => str != "Jimmy") // Filter out Jimmy
                     .Select(str => new {          // Project on to anonymous type
                         Name = str,
                         LettersInName = str.Length,
                         HasLongName = (str.Length > 5)
                     });

// Retrieve data from the enumerable
foreach (var person in people)
    Console.WriteLine("{0} has {1} letters in their name. {2}",
                        person.Name,
                        person.LettersInName,
                        person.HasLongName ? "That's long!" : ""
                      );

Steve has 5 letters in their name.
Celine has 6 letters in their name. That's long!
Arno has 4 letters in their name.

So finally we're here. You've now seen essentially how LINQ to Objects works, and using the various C# 3 features, you could pretty much reinvent it yourself if you had to. You could certainly add extra general purpose query operators if they turned out to be useful.

When Microsoft's LINQ team got this far, they organized some usability testing, had a beer, and considered their work finished. But predictably, early adopters were still not satisfied. The feedback was that the syntax was still too complicated, and why didn't it just look like SQL? All the dots and brackets were giving people a headache. So, the LINQ crew got back to business and designed a more expressive syntax for the same queries. The previous example could now be reexpressed as

var people = from str in nameData
             where str != "Jimmy"
             select new{
                 Name = str,
                 LettersInName = str.Length,
                 HasLongName = (str.Length > 5)
             };

This new syntax is called a query expression. It's an alternative to writing chains of LINQ extension methods, as long as your query follows a prescribed structure. It's very reminiscent of SQL, I'm sure you'll agree, except that select comes at the end rather than the beginning (which makes more sense when you think about it).

It doesn't make much difference in this example, but query expressions are arguably easier to read than chains of extension methods if you have a longer query with many clauses and subclauses. It's entirely up to you which syntax you choose to use—it makes no difference at runtime, considering that the C# 3 compiler simply converts query expressions into a chain of extension method calls early in the compilation process anyway. Personally, I find some queries easier to express as a chain of function calls, and others look nicer as query expressions, so I swap back and forth between the two.

The final new C# 3 compiler feature isn't something you'll want to involve in all your code, but it creates powerful new possibilities for API designers. It's the basis for LINQ to Everything, as well as some of the ingeniously expressive APIs in ASP.NET MVC.

Lambda expressions look just like lambda methods—the syntax is identical—but during compilation they aren't converted into anonymous delegates. Instead, they're embedded in the assembly as data—not code—called an abstract syntax tree (AST). Here's an example:

// This is a regular lambda method and is compiled to .NET code
Func<int, int, int> add1 = (x, y) => x + y;

// This is a lambda expression, and is compiled to *data* (an AST)
Expression<Func<int, int, int>> add2 = (x, y) => x + y;

// You can compile the expression *at runtime* and then run it
Console.WriteLine("1 + 2 = " + add2.Compile()(1, 2));

// Or, at runtime, you can inspect it as a hierarchy of expressions
Console.WriteLine("Root node type: " + add2.Body.NodeType.ToString());
BinaryExpression rootNode = add2.Body as BinaryExpression;
Console.WriteLine("LHS: " + rootNode.Left.NodeType);
Console.WriteLine("RHS: " + rootNode.Right.NodeType);

This will output the following:

1 + 2 = 3
Root node type: Add
LHS: Parameter
RHS: Parameter

So, merely by adding Expression<> around the delegate type, add2 becomes a data structure that you can do two different things with at runtime:

What's more, you can manipulate the expression tree data at runtime, and then still compile it to executable code.

But why on earth would you want to do any of this? It's not just an opportunity to write bizarre, self-modifying code that confuses the heck out of your coworkers (although that is an option). The main purpose is to let you pass code as a parameter into an API method—not to have that code executed, but to communicate some other intention. For example, ASP.NET MVC's Html.TextBoxFor() method takes a parameter of type Expression<Func<modelType, propertyType>>. You call it like this:

Html.TextBoxFor(x =>x.PhoneNumber)

This uses a whole bunch of C# 3 features. First, it's an extension method. Second, the compiler infers the two generic parameters (modelType and propertyType) from the type of the Html object you're using and the lambda expression you pass to the method. Third, the lambda expression gets compiled into a hierarchy consisting of a single MemberAccess node, specifying the model property you've referenced. ASP.NET MVC doesn't just evaluate the expression to get the property value; it also uses the AST to figure out the property name and any metadata associated with that property so that it can render a suitably annotated text box.

Now that you have lambda expressions, you can do some seriously clever things. There's an important standard interface in .NET 3.5 called IQueryable<T>. It represents deferred-execution queries that can be compiled at runtime not just to executable .NET code, but—theoretically—to anything. Most famously, LINQ to SQL (which works on .NET 3.5) and Entity Framework (the latest version of which requires .NET 4) provide IQueryable<T> objectsthat they can convert to SQL queries.

For example, assume that in your code you have something like this:

var members = (from m in myDataContext.GetTable<Member>()
              where m.LoginName == "Joey"
              select m).ToList();

LINQ to SQL coverts this into a parameterized (yes, SQL injection-proof) database query, as follows:

SELECT [t0].[MemberID], [t0].[LoginName], [t0].[ReputationPoints]
FROM [dbo].[Members] AS [t0]
WHERE [t0].[LoginName] = @p0
{Params: @p0 = 'Joey'}

So, how does it work? To get started, let's break that single line of C# code into three parts:

// [1] Get an IQueryable to represent a database table
IQueryable<Member> membersTable = myDataContext.GetTable<Member>();

// [2] Convert the first IQueryable into a different one by
//     prepending its lambda expression with a Where() node
IQueryable<Member> query1 = membersTable.Where(m => m.LoginName == "Joey");

// ... or use this syntax, which is equivalent after compilation
IQueryable<Member> query2 = from m in membersTable
                            where m.LoginName == "Joey"
                            select m;

// [3] Now execute the query
IList<Member> results = query1.ToList();

After step 1, you have an object of type System.Data.Linq.Table<Member>, implementing IQueryable<Member>. The Table<Member> class handles various SQL-related concerns such as connections, transactions, and the like, but more importantly, it holds a lambda expression object, which at this stage is just a ConstantExpression pointing to itself (membersTable).

During step 2, you're calling not Enumerable.Where() (i.e., the .Where() extension method that operates on an IEnumerable), but Queryable.Where() (i.e., the .Where() extension method that operates on an IQueryable). That's because membersTable implements IQueryable, which takes priority over IEnumerable. Even though the syntax is identical, it's a totally different extension method, and it behaves totally differently. What Queryable.Where() does is take the existing lambda expression (currently just a ConstantExpression) and create a new lambda expression: a hierarchy that describes both the previous lambda expression and the predicate expression you've supplied (i.e., m => m.LoginName == "Joey") (see Figure 3-10).

Figure 3-10. The lambda expression tree after calling Where()

If you specified a more complex query, or if you built up a query in several stages by adding extra clauses, the same thing would happen. No databases are involved—each Queryable.* extension method just adds extra nodes to the internal lambda expression, combining it with any lambda expressions you supply as parameters.

Finally, in step 3, when you convert the IQueryable object to a List or otherwise enumerate its contents, behind the scenes it walks over its internal lambda expression, recursively converting it into SQL syntax. This is far from simple: it has special-case code for every C# language operator you might have used in your lambda expressions, and even recognizes specific common function calls (e.g., string.StartsWith()) so it can "compile" the lambda expression hierarchy into as much pure SQL as possible. If your lambda expression involves things it can't represent as SQL (e.g., calls to custom C# functions), it has to figure out a way of querying the database without them, and then filtering or transforming the result set by calling your C# functions later. Despite all this complexity, it does an outstanding job of producing tidy SQL queries.

var mvcBooks = from book in new Amazon.BookSearch()
               where book.Title.Contains("ASP.NET MVC")
                     && (book.Price < 49.95)
                     && (book.Condition == BookCondition.New)
               select book;