Visual Studio Express is free to download and use, but you have to choose which language you’ll be developing with. There are separate versions for C#, C++, VB.NET, and web development, but you can have several different versions of Visual Studio Express installed on the same machine. If you have Visual Studio Professional, then you don’t have to make a choice, as it supports all of the standard .NET languages. Figure 14.1 shows a new class library being created with the VB.NET version of Visual Studio Express.

Figure 14.1. The VB.NET version of Visual Studio Express

MonoDevelop is an IDE written with the GNOME user-interface toolkit, and it works with Linux or Mac OS X.^[4] It supports C, C#, and VB.NET and is included in the Mono package for Mac OS X. Figure 14.2 shows MonoDevelop in use editing a C# project.

Figure 14.2. MonoDevelop on Mac OS X, editing a C# console project

Let’s use these IDEs to create objects we can use from IronPython.

Because IronPython is specialized for working with .NET objects, it uses the standard .NET mechanisms. This means that if an object implements IEnumerable, then we can iterate over it from IronPython; if it provides ToString, then Python will use it to produce its string representation; and so on. Even better than that, we can take advantage of some of the magic that IronPython does on our behalf. .NET methods that expect delegates can receive Python functions without the user (from IronPython) needing to be explicitly aware of the delegate. When we pass in a Python function, IronPython will wrap it with a delegate for us.

Listing 14.1 shows the code for a fairly simple class in C#. PythonClass is initialized with an integer and a callback delegate. Iterating over instances yields all the even numbers from the initial value downward, after calling the delegate we pass in originally.

Example 14.1. A C# class for use from IronPython

This is slightly odd code; the only good excuse for writing it is if you have profiled your Python code and identified a bottleneck that you can speed up by moving it into C#.

It does illustrate lots of good points for us, though. As well as allowing iteration (through the IEnumerable.GetEnumerator method it defines) and working with functions cast to the Callback delegate by IronPython for us, it also defines the operator + method (operator overloading), which allows us to add PythonClass instances together.

The last block of code in the class is the syntax for the indexer, which is the C# equivalent of Python’s __getitem__ and __setitem__ methods. A .NET class with a public indexer is, of course, indexable from IronPython. The syntax public Object this[Object index] (with a nested getter and setter like C# properties) declares a public indexer that takes an object as the index, and the getter returns an object. The setter method inside the indexer declaration receives the value being set as an implicit argument called value (it has the same type as the return type of the indexer declaration). In general, Python is my first love, but the C# syntax for properties and indexers is nicer than Python’s.

Listing 14.2 shows the same code, but this time written using VB.NET.

Example 14.2. A VB.NET class for use from IronPython

The VB.NET code is semantically almost identical to the C#, as they are very similar languages. We use the default namespace,^[5] and the iteration is provided by a list instead of yield return. The VB.NET syntax for the indexer is very similar to that of C#. It has to have the name Item and be marked as the Default property, and we also explicitly declare the value argument to the setter.

The VB.NET code is a bit less concise than equivalent Python code. My favorite example of this is the declaration of the iterator method. In Python it is def__iter__(self):. In VB.NET it is Public Function GetEnumerator() As IEnumerator Implements IEnumerable.GetEnumerator!

Figure 14.3 shows our extension class in use with IronPython (whether compiled from C# or VB.NET). A PythonClass instance is initialized with a function (that IronPython neatly turns into a Callback delegate for us) and an integer. When we iterate over it, by calling list on an instance, we can see that our callback function is called on each number before it is yielded. We can also add PythonClass instances, which call the operator + method we defined, and we can call str on them, which calls our ToString method.

Figure 14.3. PythonClass in use from IronPython with a Python callback function

We can get lots of standard Python behavior by using normal .NET mechanisms or implementing standard interfaces. To implement a Python mapping type (such as the Python dictionary), you can either use a public indexer or implement the IDictionary interface. To implement a sequence type, you can implement IList, and so on. IronPython also definesand uses several interfaces that are special.

For example, although ToString is used when str is called on an instance of our class, it isn’t used for repr. You can control the output of repr by implementing the IronPython interface ICodeFormattable. This is part of IronPython 2, and it means referencing the IronPython assemblies, which is fine if your class is going to be used only from IronPython, but not so good if you are creating a general-purpose class library. If you are happy to tie your objects to IronPython, then the easiest way of working out what interfaces you should use is to browse the IronPython source code. If you look at List.cs from the IronPython 2 source, you can see that the Python list implements the following interfaces: IMutableSequence, IList, ICodeFormattable, IValueEquality, and IList<object>.

Most of the Python-specific interfaces are defined in Interfaces.cs in the IronPython sources. You’ll see that many of the Python interfaces require you to define the Python “magic methods” that you’re already familiar with; for example, implementing ICodeFormattable means providing a __repr__ method.

In fact, we can provide standard Python features on .NET classes without having to implement these specific interfaces. This is something we’ll cover shortly.

So far we’ve done nothing that we couldn’t also have done from pure Python. In the next section we use .NET attributes to call into unmanaged code, something that we can’t do directly with IronPython.

The inability to use .NET attributes is an annoying hole in the IronPython .NET integration. Fortunately it is almost always easy to overcome by writing a small amount of C# and either subclassing or wrapping from Python.

One important attribute is DllImport,^[6] which is also known as P/Invoke (Platform Invoke), used for calling into unmanaged code.^[7] If you need to interact with native code, such as the Win32 APIs on Windows, you can decorate class methods with this attribute and use functions from native dlls.

One of the places where we use this at Resolver Systems is in our test automation framework. We use functions in the User32.dll to interact with Win32 windows. This includes making sure our forms are the foreground window (so that mouse move and mouse button events we send go to our form), closing windows, and getting the title of the topmost window (so that we can confirm that it is our form).

Some of this functionality is available in the System.Windows.Automation namespace, but this is new to .NET 3.0, and we need our test framework to be able to run on machines that have only.NET 2.0 installed.

DllImport makes it very easy to expose native functions. Listing 14.3 is an example of using Win32 APIs from User32.dll. It exposes sufficient functions for us to be able to get the title of the current topmost window.

using System;
using System.Text;
using System.Runtime.InteropServices;

namespace WindowUtils
{
 public class WindowUtils
 {
  [DllImport("user32.dll")]
  public static extern bool IsWindowVisible(IntPtr hWnd);

  [DllImport("user32.dll")]
  public static extern IntPtr GetTopWindow(IntPtr hWnd);

  [DllImport("user32.dll")]
  public static extern IntPtr GetWindow(IntPtr hWnd, uint wCmd);

  [DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Auto)]
  public static extern int GetWindowTextLength(IntPtr hWnd);

  [DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Auto)]
  public static extern int GetWindowText(IntPtr hWnd, [Out] StringBuilder
    lpString, int nMaxCount);

  [DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Auto)]
  public static extern int GetClassName(IntPtr hWnd, [Out] StringBuilder
    lpString, int nMaxCount);
  }
}

As you can see, this is a very thin wrapper around the native code we are calling. The native functions are exposed by declaring them as static methods marked as extern and with the same name and signature as the native functions. In order to do anything useful with these functions, we’re going to write some more code, but now that they’re available, we can write that code in Python rather than C#.

Listing 14.4 shows the same code using VB.NET instead of C#. The code, again, is virtually identical. The main difference is that we don’t need to declare the out parameters because VB.NET initializes them for us.

Example 14.4. A thin wrapper that exports functions from User32.dll in VB.NET

From Python, we can call these functions as static methods on the WindowUtils class. They either take or return us a window handle (as an IntPtr), and in the case of GetWindowText and GetClassName they also take a StringBuilder to put text into. Because these are C functions, they can’t just take an arbitrary string builder, but they also need to know the maximum length of the string they will be populating it with (good old manual memory management at work). In the case of GetClassName, we know that the Window class names are all less than 256 characters, so we can simply pass in a string builder initialized with a capacity of 256. Window titles can be an arbitrary length, so we have to make two calls. First we call GetWindowTextLength, which tells us the length of the string, and then we can call GetWindowText, passing in a string builder with the correct capacity.

There is a little additional complexity in the Python code that uses these functions. The full code is shown in listing 14.5, and the top-level function is GetTopWindowTitle. It gets a handle for the topmost window by calling the aptly named GetTopWindow. This turns out to be a surprisingly useless functionality on its own, because the topmost window is inevitably some invisible system window that we aren’t interested in at all. We can remedy this by using the GetWindow function and the magic GW_HWNDNEXT constant that will return us the handle of the next window. We simply iterate through all these windows until the _ExcludeWindow function finds us one that is both visible and not a system window. Having gotten the handle of the topmost visible and interesting window, we can call GetWindowText and return its title.

Example 14.5. Automation code getting the top window title by calling into native functions

You can see from figure 14.4 that all this actually works!

Figure 14.4. Printing the top window title once a second

This technique works fine where we can expose the functionality we need with a thin wrapper and then use it directly in Python. It doesn’t work so well where we want to apply an attribute to a class, or one of its members, which we want to write in Python. In order to achieve this, we still need to write some statically typed code where we can apply the attributes, but we can then subclass from Python.

You’ve seen that we can subclass .NET objects in IronPython, and we can use this to work with attributes. One place this is useful is in Silverlight for communication between IronPython and JavaScript. In the last chapter we looked at several different ways that IronPython and JavaScript can interact, but we didn’t cover every possible scenario. In particular, you may want to call into IronPython from JavaScript and have a value returned.

Doing this opens up an interesting possibility. Complex business logic in a client-side application can be written in Python and executed inside Silverlight, where it will run much faster than the equivalent code written in JavaScript. The user interface of the application, or perhaps just part of it, can be written with JavaScript UI libraries, but calling into IronPython to do the heavy lifting. The Silverlight integration could even be optional, acting as an accelerator if present, so long as you are happy to write your code in both JavaScript and IronPython!

We expose code from Silverlight to JavaScript by creating instances of types marked with the ScriptableType attribute, with methods marked with the Script-ableMember attribute, and we have the same old problem of trying to use attributes from IronPython.

We can overcome this problem by writing a very small amount of C# or VB.NET. To do this from Visual Studio we need the professional version, with Silverlight Tools^[8] installed. We can then create new class library projects for Silverlight, as you can see in figure 14.5.

Figure 14.5. Creating a Silverlight class library project with VS 2008 Pro

We can pass only the basic datatypes like strings and numbers between Silverlight and JavaScript. This restriction isn’t a problem, though, because we can pass strings to JavaScript. We can construct anything we want in the form of JSON strings, which JavaScript can eval (deserializing with a JavaScript JSON library would be better, of course).

Listing 14.6 shows a C# class with the ScriptableType attribute (from System.Windows.Browser) and a method with the ScriptableMember attribute. method takes and returns a string but actually delegates to a method called real. Because real is a virtual method, we can override it to provide the real implementation in Python.

using System;
using System.Windows.Browser;

namespace Scriptable
{
   [ScriptableType]
   public class Scriptable
   {
      [ScriptableMember]
      public string method(string value)
      { return this.real(value); }

      public virtual string real(string value)
      { return "override me"; }
   }
}

Listing 14.7 is the same class written in VB.NET. Instead of virtual, the delegated method real is marked as Overridable.

Imports System
Imports System.Windows.Browser

<ScriptableType()> _
Public Class Scriptable
   Public Function method(ByVal value As String) As String
      Return real(value)
   End Function

   <ScriptableMember()> _
   Public Overridable Function real(ByVal value As String) As String
      Return "override me"
   End Function
End Class

So how do we use this? Well, as with any other assembly, we have to add a reference to it, followed by importing the Scriptable class we want to use. We can then subclass Scriptable and provide a useful implementation of the real method that receives and returns a string when called from JavaScript. The important step is calling HtmlPage.RegisterScriptableObject, which exposes objects to the outside world. The first argument to RegisterScriptableObject is the name our object will be exposed with, and the second is an instance of a scriptable class. Listing 14.8 shows the IronPython code that makes use of our Scriptable class.

Example 14.8. Exposing a scriptable class and method to JavaScript

Having registered the object from inside Silverlight, we can now access it from the outside. The following snippet of JavaScript fetches the Silverlight control by id (one good reason to give the control itself an id when you embed it in the web page) and then calls the exposed method:

control = document.getElementById('SilverlightPlugin'),
result = control.Content.scriptable.method(value);

Note that the JavaScript calls method, which has the ScriptableMember attribute applied, rather than real, which actually does the work.

This is all fine and dandy if you have Visual Studio Professional, but it’s something of a problem if you don’t. Although Visual Studio and Silverlight Tools provide a convenient way of compiling class libraries for Silverlight, this isn’t the only way.

Compiling Assemblies Without Visual Studio

Having to have Visual Studio 2008 just to compile a few lines of C# is a nuisance. Fortunately, a C# compiler is a standard part of a normal .NET install, and we can use this directly.

The C# compiler is called csc.exe. We can pass in command-line arguments that tell it not to reference the standard .NET assemblies but use the Silverlight ones instead.

Listing 14.9 is a batch file^[9] that compiles any C# files in the current directory (*.cs) into an assembly specified by the /out argument. The /nostdlib+ /noconfig arguments tell csc not to use the standard framework assemblies, and the /r arguments add explicit references to the Silverlight assemblies we are using.^[10]

Example 14.9. A batch file for compiling .cs files into assemblies for Silverlight

This is very simple to automate as part of your build process, and it minimizes the difficulty of maintaining the small parts of a project that can’t be kept in pure Python.

We’ve been looking at creating class libraries for use from IronPython. C# and VB.NET are, like Python, imperative object-oriented programming languages. If you know one, the core concepts are similar enough that the others are easy to learn. Despite the similarities, there are also many differences, and there are times when one language is more appropriate than another. With the help of the Dynamic Language Runtime, the .NET framework is a great environment to engage in “polyglot programming.”^[11] In this section we’ve shown several good reasons to write code in C#/VB.NET for use from IronPython, whether for performance reasons or to use features of the CLR that we can’t access directly from IronPython. In fact, using .NET attributes, we can even use C# to access code written in C.

When we write .NET libraries for IronPython, they are generally usable with no special effort, but there are things that that we can do to make these objects feel more “Pythonic.”

Dynamic attribute access can be used to create a natural-feeling API, like accessing DOM elements by name on the Document class in Silverlight.

There are actually five methods that a .NET class can implement to provide dynamic attribute access.^[12] These methods are

These methods aren’t part of an interface, but instead you mark them with the SpecialName attribute.^[13] This is a standard .NET attribute (living in the System. Runtime.CompilerServices namespace), and so using it doesn’t require us to reference the IronPython assemblies.

Because you almost certainly don’t want dynamic attribute access to interfere with the normal use of your class (access to members that you have defined normally), you will probably want to implement GetBoundMember/SetMemberAfter. These are run after normal .NET lookup has been tried and are called only if the name doesn’t exist.

You can use GetCustomMember and SetMember (called before normal .NET lookups) to override normal member lookup. With GetCustomMember you can signal to IronPython that it should proceed with normal lookup by returning OperationFailed.Value.^[14] OperationFailed is defined inside the IronPython assemblies, and so you will need to reference them to use it. SetMember can signal that normal member lookup should proceed by returning a bool; True signifies that no further lookups are required and False signifies that normal lookup should continue. If SetMember returns anything other than a bool, then no further lookups will be attempted.

Listing 14.10 is a class that uses GetBoundMember, SetMemberAfter, and DeleteMember to allow you to fetch, set, and delete members.

Example 14.10. A C# class that allows dynamic attribute access from IronPython

The class DynamicObject stores attributes in a dictionary (just as they would be stored in the __dict__ dictionary on a Python object), which is exposed via the container property. The three special methods for fetching, setting, and deleting attributes manipulate the underlying dictionary. Both DeleteMember and GetBoundMember check to see if an attribute exists before attempting to fetch it or delete it from the dictionary. If the attribute is not found, then it raises a MissingMemberException (which becomes an AttributeError in IronPython) rather than a KeyNotFoundException. Returning OperationFailed.Value here would have the same effect but would require us to reference the IronPython assemblies.

The following interactive session shows our DynamicObject class in action:

>>> import clr
>>> clr.AddReference('DynamicObject')
>>> from DynamicObject import DynamicObject
>>> d = DynamicObject()
>>> d.attribute = 'an attribute'
>>> print d.attribute
an attribute
>>> del d.attribute
>>> d.attribute
Traceback (most recent call last):
AttributeError: 'DynamicObject' has no attribute 'attribute'

Because the attributes are stored in a publicly exposed dictionary, a C# application could use DynamicObject by directly working with container (an alternative would be to use an indexer and allow both indexing and attribute access). This way we have created an API that is usable from all .NET languages but uses the dynamic features of Python when used from IronPython.

Listing 14.11 is the same class implemented in VB.NET.

Example 14.11. A VB.NET class that allows dynamic attribute access from IronPython

There is a sixth method, related to dynamic attribute access, that we can also implement. In Python we determine the members available on an object by calling dir on it. This doesn’t play well with objects that dynamically create members, like the code we have just written (pure Python code has the same problem with objects that use __getattr__ for attribute access). Calling dir on a DynamicObject instance shows only the public members we have explicitly defined (plus default members that exist on all objects) and not the ones that were created by SetMemberAfter. The sixth method is GetMemberNames, and it will be used when dir is called on an object that implements it. Like the other methods, it must be marked with the SpecialName attribute, and it should return an IEnumerable (of string) that lists all of the member names on the object.

These special methods mirror the Python magic methods __getattr__ and __setattr__.^[15] There are plenty of other Python protocols that you might want to support; let’s see how you do that.

The intention is that you should be able to provide any Python magic method on a C# or VB.NET class simply by implementing it.

So if we could implement __getattr__ directly, why did we first look at GetBoundMember, SetMember, and friends? Well, there are a few exceptions—and that includes the get/set/delete member customizers that we covered in the last section. On top of this they are DLR methods rather than being Python specific. Classes that implement these methods will work with any language that runs on the Dynamic Language Runtime.

There is one more exception, too, the __call__ method, which you can implement with a Call method marked with the SpecialName attribute. The following snippet of C# shows the Call method definition:

[SpecialName]
object Call(string someArg, int someOtherArg)

In general, if there’s a .NET interface or operator method that maps onto the Python methods, you should use it. You’ve already seen how .NET indexing maps to the Python __getitem__ and __setitem__ methods. Instead of __enter__/__exit__, you can implement IDisposable and so on. This mapping of .NET interfaces and methods into Python methods is done in the source file TypeInfo.cs.

In implementing these methods we’re explicitly providing features to create objects that are easy and intuitive to use from Python. What intuitive means in a programming context is “familiar,” and this means using the techniques and patterns that are common to Python. This can be a hard goal when writing in a language that has its own idioms. There is more to the IronPython .NET integration that we can take advantage of to achieve it.

One way to create a flexible and easy-to-use API in Python is through the use of keyword arguments. Keyword arguments serve two purposes in a single mechanism. They make individual arguments to a function or method optional (supplying a default value if the argument is not passed) and make argument passing more explicit by allowing them to be passed in by name. Python also allows you to collect all arguments passed in using the *args notation.

i = SomeClass(arg1, X=value)

i = SomeClass(arg1)
i.X = value

.NET supports the latter, through the params keyword in C# and ParamArray in VB.NET, but it doesn’t directly support the concept of keyword arguments. Both VB.NET and C# have some concept of optional arguments, though, and IronPython again does magic on our behalf to make them compatible with Python keyword arguments. This means that we can write .NET classes in C# and VB.NET that accept keyword arguments when used from IronPython.

In C# we do this by marking parameters with the DefaultParameterValue attribute^[16] from the System.Runtime.InteropServices namespace. This allows us to specify a value that will be used if the argument is not passed at call time. Interestingly, we can’t omit the argument when calling from C#. Although C# allows us to create methods with parameters marked in this way, the language itself doesn’t support optional arguments.

This attribute works fine in most cases. One situation it doesn’t work in is where the parameter is of a nullable type. A nullable type is a special version of a value type (like an integer or a Boolean, which can’t normally be represented by a null) that allows null as a valid value. Under the hood .NET boxes the value, so there are performance implications, but they can be extremely useful where you need null to represent a special value. In C# you declare a variable to be of a nullable type by adding a question mark to the type declaration. A nullable integer is declared with the type declaration int?. Unfortunately we can’t use the DefaultParameterValue attribute with nullable types, but we can use the Optional attribute^[17] instead. This doesn’t allow us to specify a default value, but you can always check for null inside the body of your method and supply a default yourself.

Listing 14.12 shows a C# class with three static methods. The first takes strings as arguments but provides default values for the last two. The second method takes nullable integers as its last two arguments. It prints all three arguments it is called with, but if the second argument is null (omitted), then it is replaced with a default value inside the body of the method.^[18] The third method allows you to call it with as many arguments as you want, using the params keyword, and prints out how many arguments it is called with.

Example 14.12. Methods with multiple arguments and default arguments from C#

When this is compiled, we can call these methods from IronPython, passing in arguments by keyword. Of course in Python, and similarly in IronPython, we can always pass in arguments by name. The important difference is that with arguments marked with these attributes we can omit altogether the ones that have default values.

You can see how they behave in practice from the output from the following interactive session. Note particularly that when calling DefaultValues and OptionalIntegers, we can omit the second argument and pass in the third by keyword.

>>> import clr
>>> clr.AddReference('DefaultArguments')
>>> from DefaultArguments import Example
>>> Example.DefaultValues('first', string3='third')
First argument = first
Second argument = default
Third argument = third
>>> Example.OptionalIntegers(10, value3=2)
First argument = 10
Second argument = 100
Third argument = 2
>>> Example.MultipleArguments(1, 'two', object(), 4)
You passed in 4 arguments
Argument 0 is 1
Argument 1 is two
Argument 2 is System.Object
Argument 3 is 4

VB.NET does have language support for optional arguments. To a certain extent this makes things easier; we can just declare arguments as optional and supply a default value. These behave like arguments marked with the DefaultParameterValue attribute, which means that we can’t use them with nullable types.

In fact, the use of optional arguments is looked down on (sorry, I mean “is not recommended”) in VB.NET. The recommended technique is to use multiple overloads. You can supply one overload that takes all the arguments and alternative overloads that take fewer arguments but fill in the missing ones with default values. In this case IronPython allows us to pass in arguments by name and works out for us which overload to call. To make more than one argument optional, you will have to implement a different overload for every possible combination of arguments, and each overload must have a different type signature. Unfortunately, this makes having several optional arguments of the same type impossible just through overloads.

Listing 14.13 shows what we can do with VB.NET. The Example class here has DefaultValues and MultipleArguments methods that do the same as their C# equivalents.

Example 14.13. Optional and multiple arguments from VB.NET

Designing an API is one of the most important tasks in programming. It influences the structure and usability of your application or libraries. This is why I am a fan of test-driven development; it makes you think about the usability of your API before you think about the implementation.

We’ve been looking at ways to give an API written in C# or VB.NET that elusive Pythonic quality when used from IronPython. Of course, the easiest way to achieve this is to write your code in Python in the first place, but it may not always be possible. We have already shown how using attributes from IronPython requires the writing of at least some code in a more traditional .NET language, whether it be a stub class or a thin wrapper around native functions. In particular, writing stub classes can feel like mechanical work, ripe for automation. In fact, we can automate the compiling of these classes from text (which, after all, is what the compiler does), which means we could automate the generation of the stubs at runtime!

In the next section we use some of the APIs available in .NET that allow us to dynamically compile (and then use) code at runtime.