3.6. Dynamic Enhancements

The new dynamic functionality in .NET 4.0 allows us to work with types not known at compile time in an intuitive and easy-to-read way. The dynamic changes allow you to do the following:

• Write more readable code with fewer casting operations.

• Create new languages to utilize the .NET Framework, such as IronPython and IronRuby. Additionally, .NET's dynamic architecture allows these languages to automatically benefit from future framework advances.

• Utilize other dynamic languages and their libraries from C# and VB.NET.

• Introduce customization/scripting and debugging/querying functionality within your applications.

• Work with COM objects more easily. Microsoft Office COM is going to be around for some time whether you like it or not.

3.6.1. Can't We Do This Kind of Thing Already in .NET?

.NET has a number of classes and methods for working with types not known at compile time, such as reflection and expression trees. However, if you have spent much time with these technologies, you will known that they can make for some clunky and difficult-to-read code. As I will show you, the dynamic enhancements can make your life much easier.

Before we look at how to use the dynamic enhancements, we need to understand the difference between statically and dynamically typed languages.

3.6.2. Static Languages

In a statically typed language, such as C# or C, the compiler checks that you are using types correctly at compile time. Compilation will fail, for example, if you assign a string to an integer or misspell a variable name. Statically typed languages can catch simple syntax errors and typos during development, and as the compiler knows the types it will be working with, static languages generally run more quickly then dynamic languages, as optimizations can be performed on code.

3.6.3. Dynamic Languages

In contrast, dynamic languages, such as JavaScript, Python, Lisp, and Ruby, do not perform type checks on code until runtime. This can be a big advantage if you don't know the type of object you will be working with at compile time. These features can also make it possible to perform some interesting code tricks and reduce tedious casting operations if the language can work out how to use the type. Some developers feel dynamic languages can help them develop and prototype applications much more quickly, and can be particularly suitable for testing.

The main disadvantage of working with dynamic languages is that they of course lack compile-time checks and generally have inferior performance when compared to static languages. When working with dynamic languages, simple syntax errors and typos can stop your code from working. I think anyone that has done any web development has spent much time tracking down such an error in JavaScript, with its very helpful "object not set to an instance" message.

3.6.4. Dynamic Dangers

When working with dynamic types and the Dynamic Language Runtime (DLR), there are three considerations you should bear in mind:

• IDE support is limited.

• Generally, performance is poor (although precompilation is sometimes possible), especially on the first call to a method where the DLR has not yet cached a method call.

• Using the DLR unnecessarily creates more complexity, and more complexity is bad.

I'll cover the DLR in detail later in this chapter.

3.6.5. The dynamic Type

.NET 4.0 introduces a new type, dynamic, that allows you to tell the compiler to resolve a variable's type at runtime. Somewhat paradoxically, the keyword dynamic statically types an object as dynamic.

To declare an object as dynamic, prefix the variable name with the type dynamic:

dynamic myDynamic="Something";
Console.WriteLine(myDynamic.GetType().Name);
Console.ReadKey();

What type will be output on the second line? dynamic? object? If you said String, you are correct. I will explain why this is shortly.

3.6.5.1. Is dynamic the Same As var?

No, dynamic is very different. When the var keyword is used, the compiler infers the type of value you are using and writes the appropriate code for you when you compile the application. Variables declared using var benefit from type checks and IntelliSense, and offer better performance than their dynamic equivalents. Types declared as dynamic are evaluated at runtime and do not have these benefits.

3.6.5.2. Why Type Variables As Dynamic?

One advantage is that it can avoid some tedious casting and reflection code. For example, let's say we want to create an instance of a type using a string and call a method on it at runtime. In our example we will create a StringBuilder instance and call the Append() method on it using reflection:

object UsingReflection =
  Activator.CreateInstance(Type.GetType("System.Text.StringBuilder"));
Type ObjectType = UsingReflection.GetType();
//Append has many overloads so we need to tell reflection which type we will use
Type[] TypeArray = new Type[1];
TypeArray.SetValue(typeof(string), 0);
var ObjectMethodInfo=ObjectType.GetMethod("Append", TypeArray);
ObjectMethodInfo.Invoke(UsingReflection, new object[] { "alex" });
Console.WriteLine(
  ObjectType.GetMethod("ToString", new Type[0]).Invoke(UsingReflection, null)
);
Console.ReadKey();

By using dynamic, however, we can make this simpler and more readable (and I know which bit of code I could remember):

dynamic usingDynamic = Activator.CreateInstance(Type.GetType("System.Text.StringBuilder"));
usingDynamic.Append("Hello");
Console.WriteLine(UsingDynamic.ToString());
Console.ReadKey();

Consider using dynamic types in the following situations:

• When working with COM, dynamic types allow a more concise syntax (we saw this earlier). Let the DLR do the work figuring out how to bind your method and property calls.

• When interacting with a dynamic language such as IronPython.

• When working with objects that have changing structures, such as HTML or XML documents (we will look at this shortly).

3.6.6. System.Dynamic.ExpandoObject

ExpandoObject is a strange new beast in the .NET Framework that allows you to add and remove properties, methods, and events at runtime. The following example demonstrates how to add two new values and a method:

using System.Dynamic

dynamic MyExpando = new ExpandoObject();
MyExpando.Value1 = "new value 1";
MyExpando.Value2 = "new value 2";

MyExpando.DoSomething = new Action(() => Console.WriteLine("DoSomething called"));
Console.WriteLine(MyExpando.Value1);
MyExpando.DoSomething();

Console.ReadKey();

ExpandoObject could be used for wrapping data and making it easier to work with, and is included for interoperability with dynamic languages that support this concept.

3.6.7. System.Dynamic.DynamicObject

.NET 4.0 introduces a new class called DynamicObject that allows the definition of runtime behavior, offering a much greater level of control than ExpandoObject. It is important to note that DynamicObject is never instantiated directly, but must be inherited from.

DynamicObject is great for wrapping data and making it easier to work with, and it offers a much finer level of control than ExpandoObject. If you just need to define parameters at runtime, you will probably be adequately served by ExpandoObject. However, DynamicObject allows you full control over various operations performed.

The following example shows how to query an XML document using properties to create more readable code:

using System.Dynamic;
using System.Xml.Linq;

class Program
{
    static void Main(string[] args)
    {
        dynamic easierXML =
             new EasierXML(@"<test><node1>Alpha</node1><node2>Beta</node2></test>");
        Console.WriteLine(easierXML.node1);
        Console.WriteLine(easierXML.node2);
        Console.ReadKey();
    }

public class EasierXML : DynamicObject
    {
        private XDocument _xml = new XDocument();

        public EasierXML(string Xml)
        {
            this._xml = XDocument.Parse(Xml);
        }

        public override bool TryGetMember(GetMemberBinder binder,
                                          out object result)
        {
            string nodeName = binder.Name;
            result = _xml.Element("test").Element(nodeName).Value;
            return true;
        }
    }
}

In this example, the TryGetMember() method intercepts the call to .node1 and .node2, thus allowing you to query the XML document and return the individual nodes.

3.6.8. IDynamicMetaObjectProvider

IDynamicMetaObjectProvider is an important interface in the dynamic world that represents an object that has operations bound at runtime. Both ExpandoObject and DynamicObject implement this interface. You can use this interface to add dynamic functionality to your own classes. IDynamicMetaObjectProvider requires you to implement GetMetaObject(), which resolves binding operations (e.g., method or property invocation on your object).

3.6.9. Dynamic Limitations

When working with dynamic objects, there are a number of constraints you should be aware of:

• All methods and properties in classes have to be declared public to be dynamically accessible.

• You cannot use the DLR to create classes in C# or VB.NET. Apparently, the DLR does allow you to create classes, but this cannot be expressed using C# or VB.NET.

• Dynamic objects cannot be passed as arguments to other functions.

• Extension methods cannot be called on a dynamic object, and a dynamic object cannot be passed into extension objects.

3.6.10. Dynamic IL

You may be wondering what code the C# compiler generates when you use the dynamic keyword. Let's take a look at the IL that is generated using ILDASM for a simple console application that declares and initializes a string:

string d;
d = "What do I look like in IL";

This will generate the following IL:

.method private hidebysig static void  Main(string[] args) cil managed
{
  .entrypoint
  // Code size       8 (0x8)
  .maxstack  1
  .locals init ([0] string d)
  IL_0000:  nop
  IL_0001:  ldstr      "What do I look like in IL"
  IL_0006:  stloc.0
  IL_0007:  ret
} // end of method Program::Main

Now we will alter d to be of type dynamic:

dynamic d;
d = "What do I look like in IL?";

This will generate the following IL:

.method private hidebysig static void  Main(string[] args) cil managed
{
  .entrypoint
  // Code size       8 (0x8)
  .maxstack  1
  .locals init ([0] object d)
  IL_0000:  nop
  IL_0001:  ldstr      "What do I look like in IL"
  IL_0006:  stloc.0
  IL_0007:  ret
} // end of method Program::Main

Note how the line locals init ([0] object d) replaces locals init ([0] string d) in the dynamic example, which is probably what you expect to happen. However, let's look at another more complex example. Create a new console application called Chapter3.DynamicComplex and add the following code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Chapter3.DynamicComplex
{
    class Program
    {
        static void Main(string[] args)
        {
            TestClass t = new TestClass();
            t.Method1();
        }
    }

    public class TestClass
    {
        public void Method1() { }
    }
}

Compile the application and examine the IL using ILDASM. You will find something similar to the following:

.method private hidebysig static void  Main(string[] args) cil managed
{
  .entrypoint
  // Code size       15 (0xf)
  .maxstack  1
  .locals init ([0] class Chapter3.DynamicComplex.TestClass t)
  IL_0000:  nop
  IL_0001:  newobj     instance void Chapter3.DynamicComplex.TestClass::.ctor()
  IL_0006:  stloc.0
  IL_0007:  ldloc.0
  IL_0008:  callvirt   instance void Chapter3.DynamicComplex.TestClass::Method1()
  IL_000d:  nop
  IL_000e:  ret
} // end of method Program::Main

However, if we alter our t variable to the following:

dynamic t = new TestClass();
t.Method1();

then the IL will look very different (I have removed some of the IL to save some trees):

class [mscorlib]System.Collections.Generic.IEnumerable`1<class [Microsoft.CSharp]Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo>)
  IL_003a:  call       class [System.Core]
System.Runtime.CompilerServices.CallSite`1<!0> class [System.Core]System.Runtime.CompilerServices.CallSite`1
<class [mscorlib]System.Action`2<class [System.Core]System.Runtime.CompilerServices.CallSite,object>>::Create(class [System.Core]System.Runtime.CompilerServices.CallSiteBinder)
  IL_003f:  stsfld     class [System.Core]System.Runtime.CompilerServices
.CallSite`1<class [mscorlib]System.Action`2<class [System.Core]System.Runtime.CompilerServices.CallSite,object>> Chapter3.DynamicComplex.Program/'<Main>o__SiteContainer0'::'<>p__Site1'
  IL_0056:  callvirt   instance void class [mscorlib]System.Action`2<class [System.Core]System.Runtime.CompilerServices.CallSite,object>::Invoke(!0, !1)

Whoa, what is happening here? Well, the short answer is that calls to dynamic methods are sent to the DLR for resolution. It is time to take a look into how the DLR works.

3.6.11. Dynamic Language Runtime

The DLR is behind all the cool dynamic functionality and sits just above the core .NET Framework. The DLR's job is basically to resolve calls to dynamic objects, cache dynamic calls (making them as quick as possible), and enable interaction between languages by using a common format. The DLR has actually been around for a while, and was included in earlier versions of Silverlight. You can even view the source code behind the DLR at http://dlr.codeplex.com. Note that this version contains a number of features not present in the framework version.

When discussing the DLR, we need to understand five main concepts:

• Abstract syntax trees (ASTs)

• Dynamic dispatch

• Binders

• IDynamicObject

• Call site caching

3.6.11.1. Abstract Syntax Trees

ASTs are a way of representing code in a tree structure (if you have done any work with LINQ, you may have come across this before with the Expression class). All languages that work with the DLR represent code in the same structure, allowing interoperability.

3.6.11.2. Dynamic Dispatch

Dynamic dispatch is the air traffic control center of the DLR, and is responsible for working out what to do with dynamic objects and operations, and sending them to the appropriate binder to take care of the details.

3.6.11.3. Binders

Binders resolve classes from dynamic dispatch. .NET 4.0 currently supports the following binder types:

• Object Binder .NET (uses reflection and resolved our earlier example to type string)

• JavaScript binder (IDynamicObject)

• IronPython binder (IDynamicObject)

• IronRuby binder (IDynamicObject)

• COM binder (IDispatch)

Note that dynamic objects can resolve calls themselves without the DLR's assistance (if they implement IDynamicObject), and this method will always be used first over the DLR's dynamic dispatch mechanism.

3.6.11.4. IDynamicObject

Sometimes you will want objects to carry out resolution themselves, and it is for this purpose that IDynamicObject exists. Normally, dynamic objects are processed according to type, but if they implement the IDynamicObject interface, then the object will resolve calls itself. IDynamicObject is used in IronRuby and IronPython.

3.6.11.5. Call Site Caching

Resolving objects is an expensive operation, so the DLR caches dynamic operations. When a dynamic function or operation is performed, the DLR checks to see if it has been called already (level 0 cache). If it hasn't, then the ten most recently used dynamic methods for this call site will be checked (level 1 cache). A cache is also maintained across all target sites with the same binder object (level 2 cache).

3.6.12. IronPython

A similar process to this is used when languages such as IronPython interact with .NET. What follows is a high-level version of how the DLR processes an IronPython file:

For a detailed look at how this works with Iron Python, please refer to http://msdn.microsoft.com/en-us/magazine/cc163344.aspx.

As all languages end up being compiled into the same common AST structure, it is then possible for interaction between them.

3.6.12.1. Embedding Dynamic Languages

One use of dynamic languages that really excites me is the ability to embed them within your C# and VB.NET applications. One possibility would be to use them to define complex business rules and logic. Dynamic languages are often utilized in computer game construction to script scenarios and logic (such as how Civilization IV utilizes Python). Let's take a look at how to work with IronPython in a C# application.

3.6.12.2. Calling IronPython from .NET

The following example passes a value into a simple IronPython script from C#. Note that you should have installed IronPython from http://ironpython.codeplex.com. Now add a reference to IronPython.dll and Microsoft.Scripting.dll (at the time of writing, these don't show up on the main Add Reference window, but are located at C:Program Files (x86)IronPython 2.6).

using Microsoft.Scripting;
using Microsoft.Scripting.Hosting;
using IronPython.Hosting;

namespace Chapter3.PythonExample
{
    class Program
    {
        static void Main(string[] args)
        {
            ScriptEngine pythonEngine = Python.CreateEngine();
            ScriptScope scope = pythonEngine.CreateScope();
            string script = @"print ""Hello "" + message";

            scope.SetVariable("message", "world!");

            ScriptSource source =
              scope.Engine.CreateScriptSourceFromString(script, SourceCodeKind.Statements);
            source.Execute(scope);
            Console.ReadKey();
        }
    }
}

IronPython is already in use in two real-world applications, so let's take a look at these now.

3.6.12.3. Red Gate Reflector Add-In

Many of you will be familiar with the tool Reflector (www.red-gate.com/products/reflector). Reflector allows you to explore an assembly and view the IL code within it. C# MVP Ben Hall developed an add-in (Methodist) to Reflector that allows you to actually call the classes and methods within the type you are exploring using Iron Python. For more information, please consult http://www.simple-talk.com/dotnet/.net-tools/methodist-make-.net-reflector-come-alive-with-ironpython/.

3.6.12.4. Resolver One

One of the best-known uses of IronPython is for Resolver One (www.resolversystems.com). Resolver One is an application that allows you to work with Excel's objects using IronPython. See Figure 3-3.

3.6.12.5. Michael Foord

One of the developers on Resolver One was Michael Foord, who is author of IronPython in Action (Manning, 2009). I spoke to Michael about his experiences with working with embedding dynamic languages and IronPython.

Why should VB.NET/C# developers be interested in IronPython?

Much of the discussion here applies to other dynamic languages, including IronRuby, but Python is my particular area of expertise.

IronPython is a .NET implementation of the popular open source programming language Python. Python is an expressive language that is easy to learn and supports several different programming styles, including interactive, scripting, procedural, functional, object-oriented, and metaprogramming. But what can you do with IronPython that isn't already easy with your existing tools?

The first entry in the list of programming styles is "interactive." The IronPython distribution includes ipy.exe, the executable for running scripts or programs that also doubles as an interactive interpreter. When you run ipy.exe, you can enter Python code that is evaluated immediately and the result returned. It is a powerful tool for exploring assemblies and learning how to use new frameworks and classes by working with live objects.

The second reason to use IronPython is also the second programming style in the list: scripting. Python makes an excellent tool for generating XML from templates, automating build tasks, and a host of other everyday operations. Because scripts can be executed without compilation, experimentation is simple and fast. Python often creeps into businesses as a scripting language—but beware, it spreads.

One of the big use cases for IronPython is for embedding in applications. Potential uses include user scripting, adding a live console for debugging, creating domain-specific languages (DSLs) where rules can be added or modified at runtime, and even building hybrid applications using several languages. Python has several features, such as the ability to customize attribute access, that make it particularly suited to the creation of lightweight DSLs. IronPython has been designed with these uses in mind and has a straightforward hosting API.

There are many areas where dynamic languages are fundamentally different from statically typed languages, a topic that rouses strong opinions. Here are a few features of IronPython that make it easy to develop with:

• It has no type declarations.

• It has first-class and higher-order functions.

• There is no need for generics; it uses flexible container types instead.

• It uses protocols and duck typing instead of compiler-enforced interfaces.

• It uses first-class types and namespaces that can be modified at runtime.

• It is easier to test than statically typed languages.

• It allows for easy introspection (reflection without the pain).

• Problems like covariance, contravariance, and casting just disappear.

The best way to learn how to get the best from IronPython is my book, IronPython in Action. I've also written a series of articles aimed at .NET developers to help get you started, including

• "Introduction to IronPython" (www.voidspace.org.uk/ironpython/introduction-to-ironpython.shtml)

• "Python for .NET Programmers" (www.voidspace.org.uk/ironpython/python-for-programmers.shtml)

• "Tools and IDEs for IronPython" (www.voidspace.org.uk/ironpython/tools-and-ides.shtml)

Happy experimenting.

What does Resolver One's Python interface provide that VBA couldn't?

The calculation model for Resolver One is very different from Excel. The data and formulas you enter in the grid are translated into an interpreted language, and you put your own code into the flow of the spreadsheet, working on the exact same object model that your formulas do.

Having the programming model at the heart of Resolver One was always the core idea. When development started, the two developers (a few months before I joined Resolver Systems) evaluated interpreted languages available for .NET. When they tried IronPython, they made three important discoveries:

• Although neither of them was familiar with Python, it was an elegant and expressive language that was easy to learn.

• The .NET integration of IronPython was superb. In fact, it seemed that everything they needed to develop Resolver One was accessible from IronPython.

• As a dynamic language, Python was orders of magnitude easier to test than languages they had worked with previously. This particularly suited the test-driven approach they were using.

So the main advantage of Resolver One is that programmability is right at the heart of the spreadsheet model. IronPython is generally regarded as being a much "nicer" language than VBA. Python is a dynamically typed, cross-platform, open source, object-oriented, high-level programming language. Python was first released publicly in 1991, making it older than C#, and it is widely used in many different fields.

What do you think of the new dynamic features in .NET?

They're great, particularly for interoperating between C# and DLR-based languages. The dynamic features make this much easier.

The dynamic keyword also makes creating fluent APIs possible (like the way you access the DOM using the document object in JavaScript). This is particularly useful for DSLs.

Duck typing is one of the features of dynamic languages that simplify architecture. I doubt that the dynamic keyword will be used much for this, however, as it doesn't gel well with the way most .NET developers use traditional .NET languages.

Apart from your book (obviously!), do you have any recommended reading on Python or dynamic languages?

The Python tutorial and documentation is pretty good. Unsurprisingly, they can be found from the Python web site, at www.python.org. There is an interactive online version of the Python tutorial created with IronPython and Silverlight at www.trypython.org.

For learning IronPython, there is an excellent community resource called the IronPython Cookbook, at www.ironpython.info.

For more general Python resources, I recommend Dive into Python, by Mark Pilgrim (Apress, 2004) and Python Essential Reference, Second Edition (Sams, 2001).