When you call an instance method on a type instance, the compiler must deduce which method you are actually calling by considering things such as the instance's type, its base type (if there is one), any interfaces it and its base type may implement, and so on. As shown in Chapter 5, the steps the compiler goes through to determine which method to call can be quite complex. So how does the compiler handle the added complexity of finding an extension method to call?

Extension methods are typically imported into the current compilation unit via namespaces with the using keyword. When you use the using keyword to import the types from a particular namespace into the current scope, you also make all the extension methods implemented in static classes in that namespace available for call via the new syntax. If you don't import the namespace with the static classes implementing the extension methods you need, you can only call them as static methods using their fully qualified names. Remember that it is not required that you import a namespace in order to use the types within it. For example, you can use the System.Console type in your application without actually importing the types of the System namespace as long as you use its fully qualified name. But you typically do it for convenience. Similarly, in order to call extension methods using the instance method call syntax, you must import the namespace.

When you invoke an instance method, the compiler searches the type for the matching instance method. If the search yields no matching methods, the compiler proceeds to look for matching extension methods. It starts by searching all the extension methods declared in the innermost containing namespace of the method invocation including its imported namespaces, and if it does not find a match, it searches the next outer enclosing namespace recursively up to the global namespace as it looks for a match. If it fails to find a match, the compiler will stop with a compilation error. Conversely, if the current namespace has more than one extension method imported that matches, the compiler will issue an error complaining about the ambiguity. In such cases, you must fall back to calling the method as a static method while specifying the fully qualified method name.

Consider the following example that illustrates all these points:

using System;

public static class ExtensionMethods
{
    static public void WriteLine( this String str ) {
        Console.WriteLine( "Global Namespace: " + str );
    }
}

namespace A
{
    public static class ExtensionMethods
    {
        static public void WriteLine( this String str ) {
            Console.WriteLine( "Namespace A: " + str );
        }
    }
}

namespace B
{
    public static class ExtensionMethods
    {
        static public void WriteLine( this String str ) {
            Console.WriteLine( "Namespace B: " + str );
        }
    }
}

namespace C
{
    using A;

    public class Employee
    {
        public Employee( String name ) {
            this.name = name;
        }

public void PrintName() {
            name.WriteLine();
        }

        private String name;
    }
}

namespace D
{
    using B;

    public class Dog
    {
        public Dog( String name ) {
            this.name = name;
        }

        public void PrintName() {
            name.WriteLine();
        }

        private String name;
    }
}

namespace E
{
    public class Cat
    {
        public Cat( String name ) {
            this.name = name;
        }

        public void PrintName() {
            name.WriteLine();
        }

        private String name;
    }
}

namespace Demo
{
    using A;
    using B;

    public class EntryPoint
    {
        static void Main() {
            C.Employee fred = new C.Employee( "Fred" );
            D.Dog thor = new D.Dog( "Thor" );
            E.Cat sylvester = new E.Cat( "Sylvester" );

fred.PrintName();
            thor.PrintName();
            sylvester.PrintName();

            // String str = "Etouffe";
            // str.WriteLine();
        }
    }
}

In this example, the same extension method, WriteLine, is declared in three different namespaces (namespace A, namespace B, and the global namespace). Additionally, three types are defined, each in its own namespace: Employee, Dog, and Cat. In the Main method, an instance of each is created, and then the PrintName method is called on each instance. The body of each type's PrintName method then calls the WriteLine extension method on the field of type String, and this is where things get interesting. If you compile and execute the code, you will see the following output on the console:

Namespace A: Fred

Namespace B: Thor

Global Namespace: Sylvester

Notice that in this case the exact implementation of the extension method that gets called is governed by which namespace is imported into the namespace where the type (either Employee, Dog, or Cat) is defined. Because namespace C imports namespace A, it follows that Employee will end up calling the WriteLine defined in namespace A. Similarly, because namespace D imports namespace B, the WriteLine extension method in namespace B will get called. Because namespace E imports neither namespaces A nor B, the search proceeds out to the global namespace, which also includes an implementation of the WriteLine extension method. Finally, notice the commented code at the end of the Main method. If you uncomment these lines and attempt to compile, you will see the compiler complain that it cannot figure out which WriteLine to call because the Demo namespace imports both namespaces A and B. This example highlights the dangers of improperly using this new syntax.

How does the compiler implement extension methods? Because extension methods are just a syntactic shortcut, no modifications were needed to the runtime to support extension methods. Instead, the compiler implements extension methods completely with metadata. Following is the IL code for the ExtensionMethods.WriteLine method from the default namespace generated by compiling the previous example:

.method public hidebysig static void WriteLine(string str) cil managed
{
  .custom instance void [System.Core]System.Runtime.CompilerServices. ~CCC
ExtensionAttribute::.ctor() = ( 01 00 00 00 )
  // Code size       19 (0x13)

.maxstack  8
  IL_0000:  nop
  IL_0001:  ldstr      "Default Namespace: "
  IL_0006:  ldarg.0
  IL_0007:  call       string [mscorlib]System.String::Concat(string,
                                                                        string)
  IL_000c:  call       void [mscorlib]System.Console::WriteLine(string)
  IL_0011:  nop
  IL_0012:  ret
} // end of method ExtensionMethods::WriteLine

The bold text is wrapped to fit the page width, but what it shows you is that the method has a new attribute applied to it. That attribute is ExtensionAttribute and it is defined in the System.Runtime.CompilerServices namespace. The compiler also applies the attribute to the containing class, which is ExtensionMethods in this case. At compile time, it references this information when searching for potential extension methods to call. This is a perfect example of the power of metadata and the kinds of things you can achieve with custom attributes in metadata.

If you look at the generated IL at the call site of an extension method, you will see that the extension method is called as a normal static method.

Let's face it. There are too many companies out there that provide very little documentation of their applications' code. Many companies may have high-level design documents that essentially show the big components of the application and some vague lines joining them together to express the relationships between them. But all too often, companies are in such a hurry to get to market that once they start coding the app, and the design makes a few turns here and there, they never bother to update the original design documents. Give the product a few versions for the code and documentation to diverge further, and then you might as well just send the documents to the recycle bin. At the other end of the spectrum, you very occasionally see organizations that document everything, even up to the point of designing every piece of the application with a UML modeling tool, and so on, before coding.

Most successful projects fall somewhere in between. You typically have just enough documentation so that if the lead developer who conjured up the guts of a rendering engine meets an unfortunate fate, the next developer can come in behind him and couple the information in the documentation with the information in the code and move forward. One key to this puzzle is easily readable code, but it should also be easy-to-understand code.

What's the difference? Code that's easily readable makes the programmer's intentions at the place you are reading easy to absorb. For example, imagine that you are reading someone's code and they have a need to send some string to some log file. You could come across something like the following:

String info = "tasty information";
Logger.LogOutput( info );

For the most part, it's easy to see what is going on. Or is it? What does LogOutput do? Does it write the string to a file? Does it send it to the console? Does it send it to the debug output stream? It's impossible to tell from looking at this code. Instead, we have to look at the Logger.LogOutput method's code to understand what's going on. So one may argue that the key to understandable code is code that is easy to navigate, thus making it easy to follow.

Consider what that code might look like with extension methods in play:

String info = "tasty information";

info.WriteTo( logFile );

Instantly, the code is much easier to read. It almost reads like a written sentence. This is a technique you see used all throughout the .NET Framework. In the second line, it starts with a subject, in the middle there is a verb, and at the end is an indirect object. You could read it as if it were an imperative command given to the info object.

But what if WriteTo is implemented as an extension method? Is the code just as easy to understand as it is to read? It actually might depend on your tools. IntelliSense in Visual Studio definitely can help here because it can help you find which static class implements the WriteTo extension method. But if your favorite editor is Emacs or Vim without appropriate plug-ins, you are stuck having to determine which namespaces have been imported into the current scope, and then you must start searching those namespaces for a class that implements the extension method. Depending on the complexity of the application code and how many namespaces the current compilation unit imports, that could be a nightmare!

So the moral of the story is this. Just as with any other language feature, be sure to use extension methods correctly, sparingly, and only when needed. Just because you can implement something with an extension method doesn't mean that you always should. The entire engineering discipline is built around making engineering decisions whereby we gather information and make the best decision based upon the data, even if none of the options may be perfect.^[56]

When one first encounters extension methods, it's natural to view the feature as a way to extend the public contract of a single type or a group of types. That's an obvious conclusion given the fact that extension methods can be invoked through the instance method call syntax. However, I think it's much more effective to view extension methods as a way to provide operations that you can apply to a given type, or multiple types, in a more attractive syntactic, or general way—because again, they do not actually extend a type's contract at all.

Previously, I showed trivial examples of using extension methods to be able to call WriteLine on instances of type String. Alternatively, one could have attempted to inherit from String in order to provide the same behavior. In this case, you'll find that such a technique is impossible because String is sealed. In Chapter 13, I explain why I believe it is best to prefer sealing the types you create by default. You should unseal them only after you put in the extra thought and design (and documentation) required to make them suitable as base classes. The designers of System.String clearly had good reason to keep us from using it as a base type.

Additionally, gratuitous use of inheritance for this purpose unnecessarily complicates your design. Chapter 4 describes how you should use inheritance sparingly and how using containment is generally more flexible than inheritance. Inheritance is one of the strongest forms of static binding between two types. Overuse of that glue creates a monolith that's extremely difficult to work with.

For example, you might have the need to apply your WriteLine operation, or some other useful extension method, on a type instance that you do not create. Consider an instance of a type that's returned from some factory method, as in the following code:

public class MyFactory
{
    public Widget CreateWidget() {
        ...
    }
}

Here CreateWidget is the method that is creating the instance of the returned Widget. It's what is called a factory method. Typically, you'll have a hierarchy of type specializations derived from Widget, and CreateWidget might take some parameters telling it exactly what type of Widget to create. Regardless of that, we don't have control over how or when the object is created. Therefore, it's impossible for us to use inheritance to extend the Widget type's contract unless we also control the CreateWidget method. And even if we do, maybe that method is already published in some other assembly that's been signed and certified and cannot be easily changed. Clearly inheritance is not the correct approach to add the WriteLine functionality to types of Widget in this situation.

Extension methods also allow you to provide an operation that one can apply to an entire hierarchy of types' instances. For example, consider an extension method whose first parameter is of type System.Object. That extension method can be called on any type. You can't achieve that with inheritance. To do so would require that you have some ability to create a derived type from System.Object, say MyObject, and then somehow get every type in the CLR to derive from MyObject rather than System.Object. Clearly, there's no reason to think too much about the impossible. But just because you can create an extension method that is callable on all objects does not mean you should. Unless the extension method operates only on the public interface of System.Object, you'll most likely have some code in the extension method to determine type at run time so that you can perform some type-specific operation. Such a coding style defeats the strongly typed nature of C# and its compiler.

One of the fundamental disciplines in writing methods is that you should avoid side effects. What that means is that, if you create a method such as LogToFile and within its implementation you decide to modify some global state that is used by other components in the application, you have just introduced a potentially dangerous side effect. Side effects such as these are usually the cause of many hard-to-find bugs because, in this case, the modification of the global state might not be intuitive based on the method name.

In the same regard, try to avoid introducing side effects to your clients when they import namespaces. Specifically, it's best if you declare any extension methods in their own namespace separate from the namespace of the types they extend. Typically, the extension methods are in a nested namespace. Not giving your clients this granularity can cause confusion when the compiler attempts to look up a method for an instance method call.

Imagine, for a moment, the confusion that could come from defining your extension methods in the System namespace. There's no mechanism that keeps you from doing so. Just about everyone imports the System namespace into their code. Thus, if you define your extensions that way, most everyone will import them, and the only way to keep it from happening is for them to live with the inconvenience of not importing the System namespace at all!

If you think this is no big deal, allow me to paint a scenario. Imagine that you have an application that uses a library from Acme Widgets. The developers of Acme Widgets thought that it would be handy to introduce an extension method named WriteToLog so that you can have another debugging tool in your toolbox when using their library. Being good designers, they defined the extension method in a namespace called AcmeWidgetExtensions. Now, two versions later, you come across a library from Ace Objects that you just must have. Before making any changes to your code, you reference their assembly and include their namespace in your project. All of a sudden, your code won't build: the compiler is complaining with error CS0121 that calls to WriteToLog are ambiguous! Further investigation reveals that Ace Objects also thought it would be handy to provide an extension method called WriteToLog. Unfortunately, they defined it in the System namespace, which all your code files import. Ouch!

Thus, the moral of the story is to always define your extension methods in a separate namespace in order to allow your clients the granularity they need when importing them into their scope. Moreover, if you are offering a large set of extension methods, consider whether it would be appropriate to further partition them into multiple namespaces to offer greater granularity to your clients.

When the compiler looks for a name that matches an instance method call, extension methods are the last place it looks. This makes sense because if you have a class that already implements an instance method named WriteToLog, you don't want an extension method to replace that functionality. However, consider the following scenario.

You have an application that uses a library from Acme Widgets. To further help debug your system and to produce a rich logging mechanism, you created an extension method named WriteToLog that you can use to send information about a particular widget to a log file. Time passes, and now you have decided to upgrade to version 2 of the Acme Widgets library. But in the meantime, the creators of the Acme Widgets library decided to extend the public contract of some of their types and add a WriteToLog method because, before you implemented your own WriteToLog extension method, you sent them a feature request expressing how valuable such a thing would be. Without knowing that they added this method to their types, you recompile your code. There are no errors because the new instance method's signature just happens to match your extension method's signature exactly. But then the next time you run your application, you see some different behavior, and all of a sudden, the formatting in your log file is completely different! This happens because now the compiler prefers the instance method over the extension method. It turns out that similar bad things can happen if the type definition includes a new property with the same name as your extension method. But in that case, you get a compiler error as it complains that you are attempting to call a property as if it were a method.

The only real solution to this problem, if you ever come across it, is to switch to calling the extension method through the classic static method call syntax rather than the extension method instance call syntax. But if you're unlucky enough to have the switch happen silently, as in the previous scenario, it might be a little while before you realize that you need to start calling the extension method differently.

You might have already noticed that many of the new features added in C# 3.0 facilitate a functional programming model. You've always been able to implement functional programming models in C#, but the new language features make it easier syntactically by making the language more expressive. Sometimes, the functional model facilitates easier solutions to various problems. Various languages are categorized as functional languages, and Lisp is one of them.

If you've ever programmed using Lisp, you know that the list is one of the core constructs in that language. In C#, we can model such a list using the following interface definition at the core:

public interface IList<T>
{
    T Head { get; }
    IList<T> Tail { get; }
}

The structure of this list is a bit different from the average linked list implementation. Notice that instead of one node containing a value and a pointer to the next node, it instead contains the value at the node and then a reference to the rest of the list. In fact, it's rather recursive in nature. That's no surprise because recursive techniques are part of the functional programming model. For example, if you were to represent a list on paper by writing values within parentheses, a traditional list might look like the following:

(1 2 3 4 5 6)

Whereas a list defined using the IList<T> interface above could look like this:

(1 (2 (3 (4 (5 (6 (null null)))))))

Each set of parentheses contains two items: the value of the node and then the remainder of the list within a nested set of parentheses. So, to represent a list with just one item in it, such as just the number 1, we could represent it this way:

(1 (null null))

And of course, the empty list could be represented this way:

(null null)

In the following example code, I create a custom list called MyList<T> that implements IList<T>. The way it is built here is not very efficient, and I'll have more to say about that shortly.

using System;
using System.Collections.Generic;

public interface IList<T>
{
    T Head { get; }
    IList<T> Tail { get; }
}

public class MyList<T> : IList<T>
{
    public static IList<T> CreateList( IEnumerable<T> items ) {
        IEnumerator<T> iter = items.GetEnumerator();
        return CreateList( iter );
    }

    public static IList<T> CreateList( IEnumerator<T> iter ) {
        if( !iter.MoveNext() ) {
            return new MyList<T>( default(T), null );
        }

        return new MyList<T>( iter.Current, CreateList(iter) );
    }

    private MyList( T head, IList<T> tail ) {
        this.head = head;
        this.tail = tail;
    }

    public T Head {
        get {
            return head;
        }
    }

    public IList<T> Tail {
        get {
            return tail;
        }
    }

    private T        head;
    private IList<T> tail;
}

public static class CustomIterators
{
    public static IEnumerable<T>
        LinkListIterator<T>( this IList<T> theList ) {

for( var list = theList;
             list.Tail != null;
             list = list.Tail ) {
            yield return list.Head;
        }
    }
}

public class IteratorExample
{
    static void Main() {
        var listInts = new List<int> { 1, 2, 3, 4 };
        var linkList =
            MyList<int>.CreateList( listInts );

        foreach( var item in linkList.LinkListIterator() ) {
            Console.Write( "{0}, ", item );
        }

        Console.WriteLine();
    }
}

First, notice in Main that I am initializing an instance of MyList<int> using a List<int>. The CreateList static method recursively populates MyList<int> using these values. Once CreateList is finished, we have an instance of MyList<int> that can be visualized as follows:

(1 (2 (3 (4 (null null)))))

You're probably wondering why the list is not represented using the following:

(1 (2 (3 (4 null))))

You could do that; however, you will find that it is not as easy to use either when composing the list or consuming it.

Speaking of consuming the list, you can imagine that there are times when you need to iterate over one of these lists. In that case, you need a custom iterator, which I have highlighted in the example. The code in Main uses this iterator to send all the list items to the console. The output is as follows:

1, 2, 3, 4,

In the example, notice that the LinkListIterator<T> method creates a forward iterator by making some assumptions about how to determine whether it has reached the end of the list and how to increment the cursor during iteration. That is, it starts at the head and assumed it has finished iterating once the current node's tail is null. What if we externalized this information? For example, what if we wanted to allow the user to parameterize what it means to iterate, such as iterate forwards, backwards, circularly, and so on? How could we do that? If the idea of delegates pops into your mind, you're right on track. Check out the following revised version of the iterator extension method and the Main method:

public static class CustomIterators
{

public static IEnumerable<T>
        GeneralIterator<T>( this IList<T> theList,
                          Func<IList<T>, bool> finalState,
                          Func<IList<T>, IList<T>> incrementer ) {
        while( !finalState(theList) ) {
            yield return theList.Head;
            theList = incrementer( theList );
        }
    }
}

public class IteratorExample
{
    static void Main() {
        var listInts = new List<int> { 1, 2, 3, 4 };
        var linkList =
            MyList<int>.CreateList( listInts );

        var iterator = linkList.GeneralIterator( delegate( IList<int> list ) {
                                                    return list.Tail == null;
                                                 },
                                                 delegate( IList<int> list ) {
                                                    return list.Tail;
                                                 } );
        foreach( var item in iterator ) {
            Console.Write( "{0}, ", item );
        }

        Console.WriteLine();
    }
}

Notice that the GeneralIterator<T> method accepts two more delegates, one of which is then called upon to check whether the cursor is at the end of the list, and the other to increment the cursor. In the Main method, I am passing two delegates in the form of anonymous methods. Now the GeneralIterator<T> method can be used to iterate over every other item in the list simply by modifying the delegate passed in through the incrementer parameter.

As a final extension method example for operations on the IList<T> type, consider how we could create an extension method to reverse the list and return a new IList<T>. There are several ways one could consider doing this, and some are much more efficient than others. However, I want to show you an example that uses a form of recursion. Consider the following Reverse<T> custom method implementation:

public static class CustomIterators
{
    public static IList<T> Reverse<T>( this IList<T> theList ) {
        var reverseList = new List<T>();
        Func<IList<T>, List<T>> reverseFunc = null;

        reverseFunc = delegate(IList<T> list) {
            if( list != null ) {
                reverseFunc( list.Tail );
                if( list.Tail != null ) {
                    reverseList.Add( list.Head );
                }
            }
            return reverseList;
        };

        return MyList<T>.CreateList( reverseFunc(theList) );
    }
}

If you've never encountered this style of coding, it can surely make your brain twist inside your head. The key to the work lies in the fact that there is a delegate defined that calls itself and captures variables along the way.^[58] In the preceding code, the anonymous method is assigned to the reverseFunc variable. And as you can see, the anonymous method body calls reverseFunc, or more accurately, itself! In a way, the anonymous method captures itself! The trigger to get all the work done is in the last line of the Reverse<> method. It initiates the chain of recursive calls to the anonymous method and then passes the resulting List<T> to the CreateList method, thus creating the reversed list.

Those who pay close attention to efficiency are likely pointing out the inefficiency of creating a temporary List<T> instance that is then passed to CreateList in Main. After all, if the original list is huge, creating a temporary list to just throw away moments later will put pressure on the garbage collected heap, among other things. For example, if the constructor to MyList<T> is made public, you can eliminate the temporary List<T> entirely and build the new MyList<T> using a captured variable as shown here:

public static class CustomIterators
{
    public static IList<T> Reverse<T>( this IList<T> theList ) {
        var reverseList = new MyList<T>(default(T), null);
        Func<IList<T>, MyList<T>> reverseFunc = null;

        reverseFunc = delegate(IList<T> list) {
            if( list.Tail != null ) {
                reverseList = new MyList<T>( list.Head, reverseList );
                reverseFunc( list.Tail );
            }

            return reverseList;

};

        return reverseFunc(theList);
    }
}

The previous Reverse<T> method first creates an anonymous function and stores it in the local variable reverseFunc. It then returns the results of calling the anonymous method to the caller of Reverse<T>. All the work of building the reversed list is encapsulated into the closure created by the anonymous method and the captured local variables reverseList and reverseFunc. reverseFunc simply calls itself recursively until it is finished building the reversed list into the reverseList captured variable.

Those of you who are more familiar with functional programming are probably saying that the preceding Reverse<T> extension method can be cleaned up by eliminating the captured variable and using the stack instead. In this case, it's more of a stylistic change, but I want to show it to you for completeness' sake. Instead of having the captured variable reverseList, as in the previous implementation of Reverse<T>, I instead pass the reference to the list I am building as an argument to each recursion of the anonymous method reverseFunc. Why would you want to do this? By eliminating the captured variable reverseList, you eliminate the possibility that the reference could be inadvertently changed outside of the scope of the anonymous method. Therefore, my final example of the Reverse<T> method uses only the stack as a temporary storage location while building the new reversed list:

public static class CustomIterators
{
    public static IList<T> Reverse<T>( this IList<T> theList ) {
        Func<IList<T>, IList<T>, IList<T>> reverseFunc = null;

        reverseFunc = delegate(IList<T> list, IList<T> result) {
            if( list.Tail != null ) {
                return reverseFunc( list.Tail, new MyList<T>(list.Head, result) );
            }

            return result;
        };

        return reverseFunc(theList, new MyList<T>(default(T), null));
    }
}

The MyList<T> class used in the previous examples builds the linked list from the IEnumerable<T> type entirely before the MyList<T> object can be used. I used a List<T> as the seed data, but I could have used anything that implements IEnumerable<T> to fill the contents of MyList<T>. But what if IEnumerable<T> were an infinite iterator similar to the one created by CreateInfiniteList in the "Operation Chaining" section of this chapter? If you fed the result of CreateInfiniteList to MyList<T>.CreateList, you would have to kill the program forcefully or wait until your memory runs out as it tries to build the MyList<T>. If you are creating a library for general use that contains a type such as MyList<T>, which builds itself given some IEnumerable<T> type, you should do your best to accommodate all scenarios that could be thrown at you. The IEnumerable<T> given to you could take a very long time to calculate each item of the enumeration. For example, it could be enumerating over a database of live data in which database access is very costly. For an example of how to create the list in a lazy fashion, in which each node is created only when needed, check out Wes Dyer's excellent blog, specifically the entry titled "Why all the love for lists?"^[59] The technique of lazy evaluation while iterating is a fundamental feature of LINQ, which I cover in Chapter 16.