CHAPTER 16

Visualization and Graphical User Interfaces

GUI applications revolve around events, and F# provides a natural way to process events with functions. Graphical interfaces are often developed using visual editors, in particular to build GUIs by assembling controls. Applications, however, often need drawing capabilities for displaying and manipulating data, which requires custom programming to augment available controls. This chapter discusses how to develop graphical applications with F# and why the functional traits of the language suit the event-driven programming paradigm typical of GUIs.

In this chapter, you use the Windows Forms library. The Windows Presentation Foundation (WPF) is the presentation framework for the Windows, Silverlight, and Windows Phone platforms; it’s discussed later in this chapter. You may wonder why it’s useful to know about the old-fashioned Windows Forms toolkit. The answer is twofold: on one hand, the framework structure is shaped after a consolidated model that dominated graphical programming for decades and is still used in many other frameworks for programming GUIs, such as GTK#, a managed library for writing applications based on the GTK toolkit.

Writing “Hello, World!” in a Click

It’s traditional to start with a “Hello, World!” application, so let’s honor that and begin with a simple program that provides a button to display the magic phrase when clicked:

Even in its simplicity, the application captures many traits typical of GUI applications. After opening the namespace associated with Windows Forms, you create the form form that contains the button button, set the form and button captions by assigning their Text properties, and tell the button that it should fill the entire form.

Most GUI programming is devoted to handling events through callbacks from the graphical interface. Events are described in Chapter 11. To display a message box containing the "Hello, World!" string, you have to configure the button so that when its Click event is fired, a function is called. In the example, you pass a function to the Add method for the button’s Click event, which adds an event handler to an event source. You then add the button to the form and call the form’s Show method to display it.

Note that this code should be executed using fsi.exe. It won’t run as a stand-alone application unless you add the following line at the end:
Application.Run(form)

This line relates to the event loop of a GUI application, and it’s required to handle events such as button clicks. Moreover, if you execute the compiled program, notice that the window uses the classic Windows look and feel rather than the more fashionable look and feels featured by Windows versions since Windows XP. This can be easily addressed by adding the following call to the EnableVisualStyles static method, right after the open statement:
Application.EnableVisualStyles()

If you use fsi.exe, both visual styles and the event loop are handled by F# Interactive.

Understanding the Anatomy of a Graphical Application

Graphical applications are built on the abstractions provided by the graphical environment hosting them. The application must interact with its environment and process input in an unstructured way. User input isn’t the only kind of input received from a windowing system. Window management often involves requests to or from the application itself, such as painting or erasing a form.

Windowing systems provide a common and abstract way to interact with a graphical application: the notion of an event. When an event occurs, the application receives a message in the message queue with information about the event.

The graphical application is responsible for delivering messages from the message queue to the control for which they’re meant. A set of functions provided by the API of the windowing system supports this. This activity of reading messages from the message queue and dispatching them is known as the event loop of the application. If the loop fails for any reason, the GUI components cease to work, the application hangs, and Windows may eventually inform you that the application isn’t responding.

It’s rare for an application to program the event loop explicitly. Programming toolkits encapsulate this functionality, because it’s basically always the same. The Run method of the Application class is responsible for handling the event loop, and it ensures that messages related to events are delivered to targets within the application.

GUI programs often involve multiple threads of execution. Chapter 11 discusses threading and concurrency in more detail; for this chapter, it’s important to remember that event dispatching is a single-threaded activity, even if it may seem to be the opposite. The thread executing the event loop calls the functions and methods registered for handling the various events. In the “Hello, World!” example, for instance, you told the button to call back the function to show the message box when clicked.

An Explicit Event Loop

Software reuse has always been a priority in the world of graphical applications, because of the many details involved in realizing even simple behaviors. It’s not surprising that programming techniques favoring software reuse have always flourished in this context. You can develop a GUI application without writing a single line of code by combining existing controls into a new interface.

Articulated frameworks, such as Windows Forms, provide a significant number of reusable controls so that you can develop entire applications without needing to use drawing facilities provided by the interface. For this reason, frameworks have started to support two kinds of customers: those composing interfaces with controls and those who need to develop new controls or explicitly use drawing primitives. The following sections explore the Windows Forms framework from both perspectives: the functional nature of F# is very effective for using controls, and the ability to define objects helps you to develop new ones.

Composing User Interfaces

A control is represented by an object inheriting, either directly or indirectly, from the Control class in the System.Windows.Forms namespace. Building interfaces using controls involves two tasks: placing controls into containers (that are themselves a particular type of control) such as panels or forms, and registering controls with event handlers to be notified of relevant events.

As an example, let’s develop a simple Web browser based on the Internet Explorer control, which is a control that allows wrapping the HTML renderer (the interior of an Internet Explorer window) into an application. This example shows how to develop the application interactively using fsi.exe. Start by opening the libraries required for using Windows Forms:

Then, enable the Windows visual styles, declaring through the custom attribute STAThread that the application adopts the single thread apartment model, which is a COM legacy often required for Windows Forms applications to interact with COM components:

You need this in the example because Internet Explorer is a COM component accessible through a .NET wrapped type named WebBrowser in the System.Windows.Forms namespace, as are all the base controls offered by Windows Forms (assume that types are from this namespace unless otherwise specified).

Now you have to decide what the browser application should look like (see Figure 16-1). The bare minimum is a toolbar featuring the address bar and the classic Go button, a status bar with a progress bar shown during page loading, and the browser control in the middle of the form.

Figure 16-1. A simple Web browser application

Next, configure the elements, starting from the status bar, as shown in the following code. Add the progress bar to the status bar and configure both. The Dock property of the status bar is set to Bottom, meaning that the status bar should fill the bottom part of the form. The progress bar is resized, and then its Style property is set to Marquee, meaning that you don’t want to show any specific progress but just something moving during download:

You can now set up the browser’s toolbar, as shown in the following code. The toolbar should be in the top area of the form, so set its Dock property to Top. First add a label, the text box for typing the URL, and the button. Then, stretch the address text box and associate an event handler with its KeyPress event. This way, you can catch the Enter key and start browsing the typed URL without having to wait for the Go button. Finally, configure the Go button by setting its label and associating an event handler with the Click event:

Both event handlers set the Url property of the browser object, causing the WebBrowser control to load the given Uri. Notice how nicely and compactly F# lets you specify event handlers. This is possible because F# lets you use functions directly as arguments to Add.

You can now take care of the browser control and set its properties. Tell the browser to occupy all of the remaining area in the form left over after the toolbar and the status bar are docked, by setting the Dock property to Fill. Then subscribe to two events, Navigating and DocumentCompleted, in order to be notified by the browser when document loading starts and completes. When the browser begins fetching a URL, you show the progress bar in the status bar, setting its Visible property to true. After the document is loaded, hide the progress bar, and update the address bar so that if the user follows a link, the address shown remains consistent with the current document:

You’re almost finished with the interface. You have to tell Windows Forms that the controls are contained in the form form. Then, configure the form by setting its caption with the Text property and its size with the Size property. You call PerformLayout to update the current layout of the form, and then you can call Show to display the browser:

To compile the application rather than execute it interactively, add the following at the end, as mentioned previously:

WATCH THE APPLICATION GROW

What have you learned by developing this application? It’s clear that building interfaces based on controls requires a significant amount of code to configure controls by setting their properties. The form’s layout is also set by defining properties of controls, as you did in the browser. Moreover, an ever-increasing number of controls are available, and each provides a large number of properties. F# syntax supports this process by letting you set initial values for control properties within the call to the control constructor, and adding functions as event handlers, leading to very compact programs that are easier to understand.

CONTROL LAYOUT

Visual designers are graphical applications that allow you to design interfaces using a visual metaphor. Controls are dragged and dropped onto forms and then configured through a property window. Visual Studio provides a visual designer, although it isn’t available for all programming languages. The F# project system doesn’t feature visual design of GUIs within Visual Studio for Windows Forms or for WPF frameworks. Using the Visual Studio designer is still a useful way to explore controls and to play with their properties, however, and the generated C# fragments can be examined and used in F# code. To understand how the Visual Studio designer works, let’s design the same browser application in C#, as shown in Figure 16-2. The Visual Studio designer generates, for the form Browser, a C# file named Browser.Designer.cs containing all the C# code required to set up the interface. If you look at that file, you can see that it contains mainly the assignments of control properties similar to those you manually used to prepare the browser form.

Figure 16-2. The Visual Studio form designer

Without a visual designer for F#, there are essentially four options for building graphical interfaces in the language:

• Write the interface code by hand, as you did for the browser sample.
• Develop a visual designer that outputs F# code (although it’s a hard job), and use it.
• Use the C# visual designer, and then convert the assignments in file.Designer.cs into F#.
• Exploit the interoperability of the .NET platform by designing the interface with the designer, generating C# or another supported language and using the F# code as a library.

Leverage on existing designers and .NET interoperatibility suits many graphical applications scenarios, allowing programmers to exploit the F# data-processing expressivity and power to fuel the UI created using productivity tools. You can easily define simple interfaces with F# code, however, and the rest of this chapter shows you how to do so. You now focus on the more important part of designing graphical applications: drawing and control development.

Drawing Applications

So far, you’ve developed graphical applications based on the composition of predeveloped graphical controls, but what do you do if no graphical control suits your needs? You need to learn how to draw using the drawing primitives provided by the graphical system.

To understand how drawing works, you need to review the model behind the rendering process of a graphical application. This model really distinguishes Windows Forms from WPF; later, this chapter reviews the traditional paint-based model that still dominates the presentation systems panorama. You know already that the event-driven programming paradigm best suits graphical applications; so far, you’ve associated event handlers with user actions, but events are used by the graphical system as a general mechanism to communicate with the graphical application.

An application uses resources provided by the graphical system, and these resources are mostly windows. A window is a rectangular area on the screen, not necessarily a top-level window with buttons, a title bar, and all the amenities you usually associate with it. Windows can be nested, and they are the unit of traditional windowing systems. Windows can contain other windows, and the windowing system is responsible for ensuring that events are routed to the callbacks registered for handling the events for each window.

Windows are allowed to draw in their own client areas, and the drawing is performed through the device context, an object provided by the graphical system and used to perform the graphic primitives to draw the content. The graphic primitives issued to the graphics system aren’t retained by it; therefore, when the window is covered for some reason, the portion that becomes hidden must be repainted when it’s uncovered. Because the information required to redraw the hidden portion is held by the application owning the window, the graphical system sends a paint message to the window.

To better understand the drawing model, consider a simple graphical application that shows how to draw a curved line using the Bézier curve and canonical splines, given four control points. Start by opening your namespaces and telling Windows Forms that your form can use the current Windows look rather than the classic Windows 95 style; also, set a global flag to tell standard controls to use the new GDI+ text-rendering primitives rather than those of the traditional GDI (this code is omitted from now on, but remember to use it later; also, remember that this must be done before windows are created—otherwise, an exception is thrown):

Next, create the form and define the initial values of the control points. The movingPoint variable keeps track of the point the user is dragging, to adjust the curve:

Let’s introduce three menus to configure the application. They’re used to check features to be drawn:

Use a scrollbar to define different values for the tension parameter of the canonical spline curve:

Control points are drawn if required, and an ellipse is used to mark each of them. The function receives the device context in the form of a Graphics object; draw the ellipse by invoking the DrawEllipse primitive on it. Use a Pen to draw the ellipse—in this case, a red pen:

BRUSHES AND PENS

You’re now ready to define the function responsible for drawing in your window. You can’t assume anything about the current state of the window; thus, the paint function always draws the visible primitives¹ depending on the state of menu entries:

Figure 16-3 shows the result of the drawing all the elements. The Bézier curve, widely used in image-processing and vector applications, uses the four control points to define the start and end points of the curve and the two segments tangent to the curve at its ends. The cubic parametric curve is calculated from these points and produces the lines shown in Figure 16-3. The canonical spline, on the other hand, is a curve that traverses all the control points; the tension parameter controls how curvy the curve is.

You now want to allow users to move control points by dragging and dropping. You’re interested in mouse events—in particular, when the mouse button is pressed, when it moves, and when the button is released. Thanks to the well-defined model for rendering the application, you can update the state of your variables and ask the graphical system to issue a paint message that causes the window to receive a paint message and update the current frame.

__________

Figure 16-3. Output of the Curves application

BACKGROUND PAINTING

You define a helper function to define a circular area around a point that is sensible to your interaction. This is required in order to not require the user to pick the exact pixel corresponding to the control point:

When the mouse button is pressed, you check whether the click is over any control point. In this case, you store its index in the movingPoint variable; otherwise, the event is ignored:

When the mouse moves over the client area of the window, the mouse move event is generated. If the movingPoint member has a value other than –1, you have to update the corresponding control point with the current position of the mouse defined by the variable p:

Next, define for the window a File menu and a Settings submenu. The first features the classic Exit option, and the second shows the three checked menu items that control what the paint method should draw. You define menus by composing objects that correspond to the various menu entries. You also define the event handlers associated with each menu item. When Exit is clicked, the form is disposed. In all the other cases, you rely on the menu item’s ability to change its checked state, and you invalidate the form content to force the redraw of the window:

You’re now ready to use the functions you defined to configure the controls. Set up the scrollbar and register the controls in the form and the event handlers for the various events. Finally, start the application’s event loop and play with it:

If you’re not using F# Interactive, don’t forget to add:

Writing Your Own Controls

The Curves example from the previous section draws inside a form by handling events. This is a rare way to draw things in graphical applications, because the resulting code is scarcely reusable, and drawing on the surface of a form raises issues when additional controls have to be placed in its client area.

User controls are the abstraction provided by the Windows Forms framework to program custom controls. If delegation is used to handle events generated from controls, inheritance and method overriding are the tools used to handle them in controls.

Developing a Custom Control

To make this discussion concrete, consider a control that implements a simple button. You can use the control from C# inside the Visual Studio designer like the native button, as shown in Figure 16-4.

Figure 16-4. The F# button control used in a C# application and the Visual Studio designer

You start your control by inheriting from the UserControl class:²
namespace ExpertFSharp3.UserControls

__________

You then define the state of the control in terms of the class’s fields:

The text field contains the button’s label. As with the movingPoint variable in the Curves example, the pressed field is responsible for remembering whether the button is currently pressed, allowing the paint handler to behave appropriately. You override the OnPaint method to handle the paint event. You allocate the pens and the brush required to draw and invert the role of the border colors in order to achieve the raised effect when the button isn’t pressed and the depressed look otherwise. You also measure the size of the label string, because you’re interested in drawing the string in the center of the button. You can then draw the lines on the borders, playing with colors to obtain a 3D effect. The pens and brushes are disposed of at the end of the function:

Note the use of the colors defined in the SystemColors class: you use the system definition of colors so that the button uses the colors set by the user as display settings. Configuration is an important aspect of a user control, because it’s normally performed through a visual editor, such as Visual Studio. Well-defined controls are those that can be highly customized without having to extend the control programmatically or, even worse, to change its source code.

Now that you’ve defined the drawing procedure, you can define the behavior of the control by handling mouse events. You restrict the implementation to mouse events, although a key event handler should be provided in order to react to a press of the Enter key:

The OnMouseDown event sets the pressed member and asks the control to repaint by invalidating its content. When the mouse is released, the OnMouseUp is called, and you reset the flag and ask for repaint.

Controls are usually configured through the assignment of properties. If you annotate a property with an attribute of type Category and one of type Browsable, the property is displayed by Visual Studio in the control property box. To show this, you define the Text property, which exposes the button’s label to users of the control:

You’re now ready to test your new control by writing a few lines of F# code as:

To see your control at work in the Visual Studio designer, you must create a Windows Forms application C# project. With the default form created by the application wizard opened, right-click the Toolbox window and select the Choose Items option; then, browse for the OwnerDrawButton.dll file obtained by compiling the F# control. Now you can visually drag your F# control into the form and configure its properties using the Properties window.

VSLAB VIEWLETS

Custom controls are seen as black-box objects by the applications that host them. Several hacks are possible to handle the behavior of controls from outside (subclassing is often used on Windows), but none of them are really satisfactory. Later, this chapter discusses how this constraint is overcome by the retention-based rendering process feature in WPF.

Anatomy of a Control

As illustrated by the OwnerDrawButton control example, the structure of a graphic control tends to assume the form of a finite state automaton. Events received by the control make the automaton change its internal state, usually causing an update of its actual display.

A well-known model that describes this structure is the Model-View-Controller design pattern. As shown in Figure 16-5, the model organizes a graphical element (either an application or a single control) into three parts: the model, the view, and the controller.

Figure 16-5. The structure of the Model-View-Controller design pattern

The model constitutes the internal representation of the information displayed by the control. A word processor, for instance, stores a document in memory as part of the model, even though the entire text doesn’t fit the current visible area. In your simple button, the model is defined by the pressed and text values.

When the model changes, the view must be updated, and a rendering of the information kept in memory should be performed. Usually, the paint method corresponds to the view portion of the control. Event handlers triggered by the user form the controller of the control. The controller defines how these elements affect the model.

The Model-View-Controller pattern isn’t the only model developed to describe graphical interfaces. It captures the intrinsic nature of the problem, however, providing a good framework for classifying the elements of an interface or a control. The rest of this chapter refers to this pattern to indicate the various elements of the applications you learn how to develop.

Displaying Samples from Sensors

Now that you have reviewed the essential concepts behind programming graphical interfaces, you’re ready to work on some applications. They’re full of details typical of real graphical applications. This section presents a graphic control whose purpose is to plot samples acquired over time—for instance, from a sensor. The control has been designed to be reusable and highly configurable, providing a rich set of properties that can be set even at runtime by the application hosting it, as shown in Figure 16-6.

Figure 16-6. The GraphControl used in a test application that lets the user to change its appearance at runtime

The basic function of the control is to receive and store samples labeled with time. If c is the variable that refers to an instance of the control, the application feeds data through an AddSample method that adds a new sample to the data set, causing the control to update the displayed chart:

The next section shows how to define the control and the AddSample method.

Despite the simple interface provided by GraphControl to update the data set, the control isn’t easy to implement, because it must hold a large number of samples and provide a means to navigate through a view that doesn’t fit all the samples. Another area important to controls is configuration: users of the control want to customize its appearance to fit the needs of their application. To support these needs, you must adopt the coding conventions required to integrate the control in the Windows Forms framework so that it integrates with all the tools used to design applications.

USING FSHARPCHART

Building the GraphControl: The Model

According to the Model-View-Controller paradigm, you first define the control’s model, which should contain all the information needed to draw the control when required. You first define the type of a sample:

Samples are pairs (t, v), with t being the time at which the sample v has been read. Samples are collected in a data structure named DataSamples, whose purpose is to provide a uniform view of data; the class definition is reported in Listing 16-1. Assume that the data set is always small enough to stay in memory; this limitation can be overcome in the future by changing this class. Samples are collected into a ResizeArray and kept in the field named data. Also, use a form of run-length-encoding (RLE) compression to optimize memory usage, and use linear interpolation to make data appear continuous in order to simplify the implementation of the zoom feature. Because not all the samples are recorded, the count field is used to keep track of the number of added samples.

The AddSample method is used to add a new sample to the data set. It assumes that data come sorted with respect to time; if a sample preceding the last added is detected, it’s discarded. The Last property returns the last-added sample; you may have discarded it because it’s equal to the previous, so rebuild the sample using the lastTime field that records the time value of the last sample added.

Interpolation is done by the GetValue method, which, given a time value, calculates the corresponding value. The list of samples is searched using a binary search. If a sample matches the given time, it’s returned; otherwise, the interpolation is performed.

The last operation implemented by DataSamples is FindMinMax, a method that computes the minimum and maximum values of the data in a given interval. You can initialize the values for minimum and maximum, as well as a stride to use to do the search. The stride is useful in conjunction with zoom, because the number of samples that can be displayed is finite, and the rendering procedure must subsample when zooming out.

Building the GraphControl: Style Properties and Controller

Listing 16-2 shows the code for the GraphControl class except for the OnPaint drawing method, which is shown in the next section. This control exhibits the typical structure of a graphic control; it features a large number of constants and fields that serve configuration purposes. The class inherits from UserControl, which is the base class of Windows Forms controls, and it contains a field named data of type DataSamples that represents the data shown by the control. The appearance is controlled through properties, fields, and constant values; for instance, the axis color is controlled by the pattern:

The AxisColor property lets the control’s host change the color of the axis color displayed by the control, because properties are part of the controller of the control; thus, when the setter is invoked, you call the Invalidate method to ensure that a paint message is sent to the control so that the view is updated. Note that a fully fledged control might read defaults from a store of user-specific configuration properties.

Listing 16-3 includes the remaining portions of the GraphControl class corresponding to the controller part of the Model-View-Controller paradigm. Samples are added through the AddSample method (AddSampleData generates random samples to display inside the control). This method adds the sample to the inner DataSamples object and updates the values of two fields meant to store the minimum and maximum values recorded for samples; both of these values are used in the display process. Because the model of the control changes, you need to update the view, and you invalidate the control as you did for properties.

OVERRIDING VS. DELEGATION

Let’s look at how to handle the mouse-move events and the use of the mouse wheel. When the wheel of the mouse is scrolled, the control adjusts the scale factor to zoom in or out of the current view. To understand how this method works, you need to know how to decide which portion of the data is made available through the view of the control. You use two fields: initView and visibleSamples. Because you can’t assume that all the samples fit in the control’s display, the former indicates the time (in the time scale of the samples) corresponding to the leftmost visible value, and the latter indicates the number of time units in the unit scale of samples that should be visible. Zooming is performed by changing the density of time units to be displayed inside the viewport.

The last method in Listing 16-3 is GetTime: it converts the time unit of samples in microseconds using a scale factor that is one of the configuration properties made available by the control.

Building the GraphControl: The View

The view of the GraphControl is entirely contained within the OnPaint method, which is invoked when the GUI needs to repaint the content of the control or when an invocation of the Invalidate method occurs. Listing 16-4 shows the full code for this method. Programming graphical controls can get complicated, and often the code is factorized further using functions.

The OnPaint method begins computing some information, such as the rectangles containing the string with the values to be displayed. The dimension of a string depends on the font used for display and the particular device context used to render it. You rely on the MeasureString method of the Graphics object you received from the GUI. You compute the plotBox rectangle, which represents the area where you draw the data; it’s obtained by removing from the dimension of the control that the margins specified in the configuration and the space required by the labels, if visible. Later, set an appropriate coordinate system on the device context so that the drawing primitives render in this new system:

You translate the origin of the coordinate system in the lower-left vertex of the margins rectangle. You also flip the y axis by setting a scale transform that inverts the direction, multiplying y coordinates by –1.0f; in this way, you obtain a coordinate system oriented as in mathematics. Coordinate transformation is supported by Windows Forms on the Graphics object: all the coordinates specified in the drawing primitives are affected by a transformation matrix stored in the device context. Once set, a transformation of the library takes care of the calculations necessary to rotate, translate, and scale all the objects.

After clearing the background using the Background color property, you draw the various lines, such as the axes and the labels, depending on the configuration settings specified by setting the control’s properties. This is the typical structure of a paint method, in which the model is tested to decide what should be drawn and the style to be used.

The drawing of the data samples is controlled by the timePerUnit and pixelsPerUnit variables, and then the inner recursive function drawSamples selects the visible samples and uses the DataSamples object to compute results. You rely on the ability of the DataSamples class to interpolate data and to not have to deal with discrete samples.

The core business of the paint method is often simple (having paid attention when you defined the model and the controller of the control); it quickly becomes entangled in testing all the configuration properties to determine how the control should be rendered.

DOUBLE BUFFERING

Putting It Together

The following code uses the control that defines the application shown in Figure 16-6:

The form uses a SplitContainer control to define two areas, one for GraphControl and the other for a PropertyGrid control. A timer object is used to add samples periodically, and you use the AddSample method to add random samples to the control.

THE PROPERTYGRID CONTROL

Creating a Mandelbrot Viewer

Fractals are one of the diamonds of mathematics. They show the beauty of mathematical structures visually, which allows nonexperts to see something that is often hidden by formulas that few really appreciate. The Mandelbrot set is one of the most famous fractals. This section shows how to develop an application to browse this set. The result is shown in Figure 16-7.

This application adopts the delegation programming style, subscribing to events rather than using inheritance to override the behavior of a component. This allows you to develop the application interactively using fsi.exe. This is a good example of how effectively you can use F# to develop an application interactively while retaining the performance of a compiled language, which is extremely important in such CPU-intensive tasks as computing the points of the Mandelbrot set.

Figure 16-7. The Mandelbrot viewer in action

Computing Mandelbrot

First, you need to be familiar with the math required to generate the Mandelbrot set. The set is defined over the set of complex numbers, which is an extension of the real numbers, allowing computation of square roots over negative numbers. A complex number has the following form, where a and b are real numbers and i is the imaginary unit (by definition, i² = –1):

Using standard algebraic calculations, you can define the sum and product of these numbers:

Because you have two components in the definition of the number, you can graphically represent complex numbers using a plane.

This Mandelbrot viewer shows a portion of the complex plane in which each point in the plane is colored according to a relation that defines the Mandelbrot set. The relation is based on the iterative definition:

A complex number belongs to the Mandelbrot set if z_n converges for n. You can test each number c in the complex plane and decide whether the number belongs to the Mandelbrot set. Because it’s not practical to perform an infinite computation to test each number of the complex plane, there is an approximation of the test based on a theorem that if the distance of z_i from the origin passes 2, then the sequence will diverge, and the corresponding z₀ won’t belong to the set.

The code to compute membership of the Mandelbrot set is:

You can create a simple visual representation of the Mandelbrot set by coloring all the points belonging to the set. In this way, you obtain the black portion of Figure 16-7. How can you obtain the richness of color? The trick is to color points depending on how fast the sequence reaches the distance of 2 from the origin. You use 250 colors and map the [0, maxit] interval to the [0, 250] discrete color interval.

Setting Colors

In order for the Mandelbrot viewer application to have appealing coloring, you need to produce some form of continuity in the color variation in the chosen range. Use an array of colors to store these values, but you need a procedure to fill this array so that colors change continuously.

Colors in the Red Green Blue (RGB) space used in graphics libraries are known to define a color space that isn’t perceived as continuous by human vision. A color space known to be more effective in this respect is the Hue Saturation Value (HSV), in which a color is defined in terms of hue, color saturation, and the value of luminance (see Figure 16-8). This model was inspired by the method used by painters to create colors in oil painting.

Figure 16-8 shows a typical geometric representation of the two color spaces. In the RGB color model, the three axes represent the three base colors varying from black to the full color; in the HSV space, the angle is used to indicate the hue, the distance from the axis of the cone represents the saturation, and the value corresponds to the height of the point inside the cone.

Figure 16-8. The RGB (left) and HSV (right) color spaces and their geometric representation

You can define a conversion from RGB color space to HSV, and vice versa. Listing 16-5 shows the F# functions performing the conversions between the two models. These functions assume the three components R, G, and B are in the interval [0, 1] rather than integers between 0 and 255.

To let users choose the coloring of the set, you create an array of 10 functions, given an integer between 0 and 250, for a corresponding color. The default color function is based on the HSV color model; it uses the input parameter to set the hue of the color, leaving the saturation and luminance at the maximum values. The other functions use the RGB color space, following directions in the color cube. You use the createPalette function to generate the color palette that is used while drawing fractal points; the palette mutable variable holds this palette. Listing 16-6 shows the code that deals with colors. The pickColor function is responsible for mapping the iteration it, at which the computation of the Mandelbrot set terminates given the maximum number of iterations allowed, maxit.

Creating the Visualization Application

You’re now ready to implement the graphical application. The basic idea is to map a rectangle of the complex plane in the client area of your form. Each point of your window corresponds to a complex number and is colored according to the value computed by the mandel function.

A typical value for maxit is 150 for the initial rendering of the Mandelbrot set, although it should be incremented when zooming into the fractal. The total computation required to generate an entire image is significant; therefore, you can’t rely on the main thread to perform the computation of the various points. It’s important to recall that event handlers are invoked by the graphical interface’s thread, and if this thread is used to perform heavy computations, the application windows won’t respond to other events.

You introduce a thread responsible for performing the computations required by the Mandelbrot set so that the GUI thread can continue to handle events. (Chapter 11 discusses threads in more detail.) Here, you use shared memory to communicate the results between the threads by using a bitmap image in memory, referenced by the bitmap variable, which is updated by the thread performing the computation task and read by the GUI thread when the form must be painted. The bitmap is convenient, because you need a matrix of points to be colored, and the device context is designed to avoid pixel coloring to be device independent (it doesn’t provide a SetColor(x, y) operation). To avoid race conditions, use the lock function to guarantee exclusive access to the bitmap shared between the two threads. Chapter 11 looks at this in more detail. The thread responsible for the set computation executes the function:

Use dx and dy variables to map the x and y coordinates of the bitmap into the complex plane. Then invoke the filler function responsible for performing the calculation. There are different possible filling strategies to compute the colors of the set; the straightforward approach is left to right and top to bottom, implemented by the linearFill function:

Another typical filling strategy is to gradually refine the set by computing points in blocks and filling the blocks of the appropriate color; then, the missing points are computed by refining the block size. Using this strategy, you can provide a quick preview of the fractal without having to wait for the entire computation. The blockFill function implements this strategy:

The variable fillFun is used to store the current filling function:

You clear the bitmap by obtaining a device context to the bitmap and clearing it. The global variable bitmap is used to access the image from the code; this is an effective choice to speed up the development of the application. This technique can be a problem from a software engineering standpoint, however, because the program is less modular, and the mutable state isn’t encapsulated:

To refresh the application form while the fractal computation is ongoing, use a timer that triggers a refresh of the form every tenth of a second (the fully qualified name for the Timer class is needed to disambiguate between System.Threading.Timer and System.Windows.Forms.Timer). The paint function draws the bitmap that is updated by the worker thread:

The drawMandel function is responsible for starting the rendering process:

Creating the Application Plumbing

Now that you’ve defined the architecture of the application, you can define the graphical aspects, the form, and the menus, as well as how users interact with the application. The code is similar to the previous applications, as shown in Listing 16-7. Note two aspects: the rect variable contains the current selection, and it’s drawn as a rectangle filled with transparent white; the application uses the clipboard to store and retrieve the coordinates of a particular fractal view or save the current state of the drawn bitmap. When Ctrl+C is pressed, a small XML document similar to the following is saved to the clipboard:

The saved parameters are the most compact representation of the current view, and they are loaded back if Ctrl+V is pressed; this way, you can save the state of navigation. You save and read text from the clipboard using the Clipboard class’s SetText and GetText methods. When the Ctrl+Shift+C key sequence is pressed, the current bitmap is copied to the clipboard using the SetDataObject method; the bitmap can be pasted in any program capable of pasting images.

The selection rectangle is updated by the mouse event handlers. You obtain the zoom facility by setting the bounds of the complex plane defined by the variables tl and br.

Use the program’s last statement only if the application is compiled; with fsi.exe, it must be omitted. As already noted, the plumbing code is dominated by setting up menus and configuring the application form; the rest contains the event handlers that are registered with the various controls.

Windows Presentation Foundation

Microsoft introduced a new presentation framework called Windows Presentation Foundation (WPF) for programming graphical applications on the Windows platform with Windows Vista. It’s not merely a new framework, such as Windows Forms; rather than another layer on top of the traditional Win32 presentation system, it’s a complete new rendering system that coexists with the traditional one. More recently, Windows 8 introduced a new UI framework, the foundation and architecture of which has its roots in WPF and requires a compiler to output a new binary format called winmd.

At first sight, a WPF application doesn’t differ much from a Windows Forms application. Here are few lines of code to open a window from F# Interactive:

The application has an event loop (see the sidebar “WPF Event Loop”) hosted by F# Interactive. With a few (very important) differences, you’ve created a window hosting a huge button that closes it when clicked.

WPF EVENT LOOP

It’s reassuring that events are still there, even within a framework based on a different programming paradigm. Moreover, you still create user interfaces by composing graphical controls and attaching to their event handlers. The button is the only control hosted by the window! You don’t access a Controls property of the window to add the button to a list of children controls. Instead, you set all the window content with a button, resulting in a huge button that occupies the window’s entire client area.

WPF’s layout model is completely different from what you’ve grown accustomed to using Windows Forms: objects are positioned by the container and don’t have Top and Left properties to explicitly set a control position.

Layout is only one of many aspects of WPF that diverge from the traditional structure of graphic toolkits; this section introduces the most relevant. Look for more information about the framework in dedicated publications. The most important changes introduced by WPF are:

• Retention-based, device-independent rendering
• XML-based composition language
• Dependency properties and animations

When GUIs Meet the Web

The long-established model of GUI toolkits has its roots in a world very different from today’s. Several assumptions made by these programming models are still true, but others no longer hold. This is an important point that you must always consider when working with WPF after any other graphical toolkit.

One of the authors was introduced to graphical programming during the 1980s, using Turbo Pascal, and was surprised that drawing primitives weren’t retained by the system like console characters. The situation, however, was better than it had been a few years before, on a MS-DOS PC with graphics: the screen buffer was owned by a single application, so redrawing was needed to remove a primitive and restore its background. With windowing systems, the user experience improved significantly for the user, but the situation worsened for the application programmer: the application’s main control must be delegated to the graphical toolkit, and everything becomes an event. Events significantly optimize resource usage, because applications are notified only when needed so that a user can execute multiple applications simultaneously, but events are also used to optimize drawing resources through the paint architecture discussed previously. As you’ve seen, the GUI system sends the paint message to a window whenever its content needs to be updated; therefore, primitives aren’t retained by the windowing system. It’s up to you to issue them as necessary.

In essence, the familiar paint-event-based paradigm you’re used to exists because of the need to save memory when it’s scarce; this is a form of Kolmogorov-style compression, in which a program generates information instead of storing it. As illustrated by Web browsers, today’s computers have enough memory to change this underlying assumption and revise the original decision of who is responsible for retaining graphics primitives. With WPF, Microsoft decided to radically change the toolkit model and define a foundation based on the notion of graphics-primitive retention.

This example shows how a typical drawing is performed by a WPF application:

At first, this example doesn’t differ much from the one showing a button in a window: you create a Canvas object, which is a container for multiple UI elements, and add two graphic objects. In the Windows Form world, you’d think of two graphic controls used for drawing, although they wouldn’t blend, because each traditional window is opaque. In fact, these two graphics primitives are represented by two objects and are retained by the presentation system; pick correlation and event dispatching are guaranteed by the graphical toolkit.

Note that the ellipse is an outline, so the click event is raised only by clicking the outline; when the fill brush is specified, you can change the fill color by clicking inside the ellipse. Another relevant aspect is the positioning of an element: there is no Left or Top property, because it’s the responsibility of the container (in this case, the canvas) to lay out contained objects.

Drawing

An application can draw any shape by adding graphics primitives to a WPF window’s visual tree. Because the system retains these primitives, you don’t need to implement an event handler for the paint message: it’s up to the graphical toolkit to render the tree in the appropriate way whenever needed. Problems such as visible primitives and efficient drawing fade away quickly and are dealt with by the system; nevertheless, you’re still responsible for using the visual tree.

Because graphics primitives are organized in a tree structure, it’s natural to use an XML-based language to initialize this tree with the window’s initial state. WPF introduced the XAML language for this purpose; it recalls HTML, though it was designed for defining rich UIs instead of hypertext. The drawing example shown in the previous section can be expressed using the XAML markup in Listing 16-8.

The markup syntax is a wonderful tool for compacting several programming-language statements into a single line, because properties can be set as attributes. XAML adopts a well-defined bridge between the XML elements and the corresponding .NET classes and objects. Tag names are mapped onto classes and implicitly define instances, attributes are used to assign objects’ properties, and attributes containing a dot are interpreted as static properties (for example, Canvas.Left and Canvas.Top). Each node may have a name, which is useful for retrieving a reference to the node; in this case, the ellipse element is named Circle.

LOADING XAML INTO F# APPLICATIONS

XAML has been rightly touted as separating the world of business logic from that of UI design, offering direct control to graphic designers through XAML and allowing programmers to rely on names to instrument graphical controls. XAML and the names of graphical elements are the contract between programmers and designers, allowing for concurrent work on GUI-application development. This approach has proven successful for the Web and shows that hardware is powerful enough to shift toward a more programmer-friendly paradigm.

XAML can be generated and manipulated from a number of graphical editors, including Visual Studio and Microsoft Expression Blend. Even though full support isn’t yet available for developing WPF applications using F#, it’s easy to load XAML-based UIs into F# applications; this largely addresses the lack of graphical environments for developing GUIs (see the sidebar “Loading XAML into F# Applications”). Assuming that the XAML in Listing 16-8 is saved in a file named window.xaml, you can use it from F# (assuming the appropriate references and the loadXamlWindow function are defined) as:

You use the FindName method to find a reference to the ellipse object; this is how you can refer from F# code to elements defined in the XAML markup, unless you use the XAML type provider, in which case you simply access the Circle property on the type generated by the provider. Because you can define animations in markup and connect to user actions (for example, sliding or collapsing menus can be defined entirely in XAML), the number of statements needed to connect the F# state to the visual tree is small—drastically smaller than the amount of code needed to do GUIs in Windows Forms. Sometimes it may be useful to navigate the entire visual tree in which WPF stores the graphics primitives; the VisualTreeHelper class features navigation methods useful for doing this.

Drawing primitives refer to the coordinates of some coordinate space. Traditionally, computer programs relied on pixels, even though Win32 API supported different coordinate systems from the beginning. WPF features vector graphics, supporting affine transformations of the coordinate system (translation, rotation, scaling, and skew); the use of a pixel-based system seemed obsolete in a world in which the dots per inch (dpi) of display devices changes significantly. The adopted unit is 1/96^th of an inch (in honor of the traditional monitor resolution), whatever the target display device, including printers.

You can apply transformations to any subtree of the visual tree, transforming all the primitives, including those issued by contained controls.

Let’s consider the XAML example shown in Listing 16-9. It uses a container that stacks graphical elements horizontally to lay out three elements: a block of text, a text box, and a line. Nodes may also have explicit names, such as the TextBox node here. The application’s output is shown in Figure 16-9, where the elements are rotated according to the indicated transformations: the container is laid out rotated 45 degrees counterclockwise, and the text box is rotated 90 degrees clockwise.

Figure 16-9. F# application demonstrating the XAML

Transformations are implemented using matrices, as already discussed for Windows Forms; you can use them to obtain nontrivial transformations of the coordinate space. These transformations affect the subtree rooted in the node to which they’re applied and can be programmatically accessed through the LayoutTransform and RenderTransform properties. The former affects the layout computation, and the latter affects the final rendering, possibly overflowing the container node.

Drawing involves brushes and pens, which are similar to their Windows Forms counterparts.

Controls

The ability to instantiate and use existing controls is one of the pillars of user interfaces. In the earlier discussion of the difference between writing components and combining them, it was evident that a control in a traditional framework is opaque: the container allocates a rectangle and delegates event handling and drawing to the window procedure coordinated by the event loop.

In WPF, things are less clear-cut, which opens new horizons to graphical developers who appreciate the visual tree and the different role played by controls. To appreciate the extent of this statement, let’s modify the example from Listing 16-9 by enclosing the StackPanel element in a Button element (add a <Button> tag immediately after <Canvas> and </Button> immediately before </Canvas>). The resulting window looks like the one shown in Figure 16-10: you’ve nested two controls, which is an almost impossible task in traditional graphical toolkits.

The designers of the WPF framework must have considered problems that never came up with traditional toolkits, given the system’s ability to retain graphics primitives in the form of a tree. You can think of a button as a background shaped like a button, with the label in the middle; it can be defined as a subtree of the primitives needed to draw its borders and a placeholder in the middle to insert the label. In an opaque system, the content could be another rectangle but nothing more; in a retention-based system, you can replace the label node in the tree with an arbitrary, complicated subtree.

Figure 16-10. Nested controls in WPF

You can display the visual tree of any WPF element using the static methods of the VisualTreeHelper class. The following F# function prints to the console (or in the F# Interactive tool window) the visual tree associated with a WPF element:

The function implements a recursive visit to the visual tree rooted in the given element. If you pass the instance returned by loadXamlWindow, you get an idea of how many primitives are issued by the various controls you’re using in the window. If you pass the instance of a button, perhaps found using the FindName method as you did earlier, you obtain output similar to:

ContentPresenter is the root of the subtree dedicated to the button content; the rest defines the graphics primitives surrounding the node. The ButtonChrome node refers to a class used to implement a specific theme; it’s even possible to change the definition of the button without affecting its logic or its content. You can find more information about this ability of WPF in textbooks or online.

EVENTS AND WPF

The type expected by the ShowVisualTree function is DependencyObject, which returns to the notion of dependency—a pervasive concept deeply entrenched within the framework. Dependency is a relation among two or more elements; in its asymmetry, it expresses something whose nature depends on something else. Dependency properties are a programming pattern used across WPF to define properties that you can listen to for change. If w is a Window object obtained from parsing the XAML contained in Listing 16-9, you can read and set the Text property of the TextBox object from F# as:

The Text property behaves as a standard property, and you can use it programmatically as you’re used to doing with all properties. Text is a dependency property, and any observer in the code can listen for changes. You can detect its change as:

You use the conventions used by TextBox implementers to access the descriptor of the Text dependency property called TextProperty and rely on appropriate data structures to register your callback. Whenever the text-box value changes, a message box is displayed.

Implementing a dependency property is easy: in the class, you define a static field describing the property; then, the property must be registered in a global structure hosted by the DependencyProperty class. You then add a standard .NET property using the GetValue and SetValue methods—inherited from the DependencyObject class, which must be an ancestor of the class defining the property—hiding the implementation details. Supported callbacks are registered to inform the framework about the observable events on the property (such as the property-value change).

At first, this may seem unnecessarily verbose: why is it useful to be able to observe when a property changes its value? The answer lies in WPF’s ability to support the animation of virtually any element property and the fact that dependency properties are the building blocks of the animation engine. Animations often correlate properties of different visual objects. For instance, think of an object and its shadow: whenever the object’s position changes, its shadow must follow. Dependency properties provide a pervasive infrastructure to propagate property changes across visual tree nodes in a consistent way. To exemplify this idea, let’s change Listing 16-9 again by replacing the TextBox element definition with:

The expression in curly braces is a data-binding expression, which is an expression evaluated by WPF engine to set the Text property of the TextBlock node. If you run the example, the text block updates its content when the text box changes its value. WPF defines several forms of data binding to declare how dependency properties must be updated without explicit coding; you can define several automatic actions using these mechanisms.

In WPF, controls are a way to instantiate subtrees and attach predefined behaviors to nodes and properties. They’re less opaque in WPF than before; nevertheless, if a change in the visual tree disrupts the assumptions made by a control’s logic, a control may fail. For this reason, controls in WPF offer placeholders for inserting content, as you’ve seen with buttons, as well as a means to change the control’s appearance while preserving the essential traits of the visual subtree that make its inner code work.

Bitmaps and Images

Vector graphics, after being abandoned in favor of raster graphics, have again become popular, because they let you define graphics that can be scaled without the annoying pixelated effects typical of raster graphics. Vector primitives are exact, and the rendering engine can compute the value of any number of pixels affected by the primitive. If you zoom a line, you can compute all the pixels of the line without any loss in detail. If the line is contained in a bitmap, this isn’t possible: the zoomed line contains artifacts introduced by the algorithm that resized the image.

It’s impossible to imagine a user interface without images. What, however, does it means to work with pixels in a world of abstract coordinates where points are considered sizeless entities, as in geometry? An image’s size is defined in the application space, and WPF is responsible for the rastering process and for generating enough pixels at a given resolution. A 100×100 image at 96dpi uses the same number of pixels on a standard monitor, or more pixels if the device has a higher dpi resolution. This concept isn’t unfamiliar: in the Mandelbrot example, you used a bitmap to set pixel colors, because of the lack of a SetPixel method on the Graphics object representing the device context for drawing. WPF, however, defines its own set of classes to deal with images, featuring a slightly different set of properties and capabilities.

To exemplify the use of images with WPF, let’s consider a simple but useful application: a batch-size conversion for JPEG images. The application is a command-line tool that converts a directory containing hi-res pictures into a mirrored version containing resized versions of the same pictures for a specific resolution. With this tool, it’s easy to convert thousands of photos into versions suitable for a photo frame, thus saving space. The conversion process preserves JPEG metadata information such as tags, which may be useful after conversion.

The two main functions are shown in Listing 16-10. transformFile is responsible for transforming a single JPEG image and saving it into a new file; transformDir performs a recursive visit to a directory tree and builds a mirror tree in the destination directory, relying on the first function to convert the image files.

The image transformation relies on JpegBitmapDecoder and JpegBitmapEncoder, two classes dealing with the JPEG format. These classes support a streaming model for decoding and encoding images, so you read information about the first frame (an image may contain more than a single frame) to ensure that all the metadata is read from the source image. The desired output size is passed in the form of a screen resolution through the width and height arguments, and a best-fit policy is used to define the size of an in-memory bitmap instance of the class BitmapImage. The image resize is performed by asking the in-memory image to load data from the source file and fit the desired size; this way, you avoid allocating the full image in memory and performing the resize operation while reading pixels from disk. You then use the encoder to save the image, with the desired quality, in the new file, along with the metadata and the creation-time and change-time information.

Start the transformation process by invoking the transformDir function with the JPEG compression quality, the target screen size, and the input and output folders. The following lines assume that you’re converting summer photos for a frame supporting 1024×768 resolution:

Bitmap manipulation can be very tricky using WPF, and some tasks are more difficult than they are using Windows Forms. In particular, it’s easy to obtain a device context to a bitmap and draw using GDI functions. You can raster WPF primitives into a bitmap, but many programmers prefer to refer to both libraries and to convert images from one framework to the other, in an attempt to benefit from both worlds.

Final Considerations

WPF isn’t just another framework built on top of the traditional Win32 and GDI rastering engine: it’s a new presentation system designed with a different programming paradigm in mind. This section introduced some important ideas behind this new presentation system and how you can use it from F# applications. The content is far from exhaustive, and several topics have been left out or only mentioned in an attempt to convey the core ideas as a starting point for further investigation. The literature about WPF is very rich, and we encourage you to read more specific material.

Summary

Event-driven programming is the dominant paradigm of graphical applications. Although object-oriented programming is used to build the infrastructure for graphical frameworks, events are naturally expressed in terms of calling back a given function. F# is effective in GUI programming, because it supports both paradigms and provides access to full-featured frameworks, such as Windows Forms.

This chapter covered the basics of GUI programming. It introduced controls and applications, including details typical of real-world applications. The presentation was based on the Windows Forms framework, but many of the ideas discussed can be easily transposed to other graphical frameworks: the fundamental structure of this class of applications is mostly unchanged. Finally, we introduced WPF and discussed differences between it and Windows Forms.