Intent

To create a function that itself creates functions, allowing us to synthesize behaviors on the fly

Overview

Sometimes we’ve got a function that performs a useful action, and we need a function that performs some other, related action. We might have a vowel? predicate that returns true when a vowel is passed in and need a consonant? that does the same for consonants.

Other times, we’ve got some data that we need to turn into an action. We might have a discount percentage and need a function that can apply that discount to a set of items.

With Function Builder, we write a function that takes our data or function (though, as we’ve seen, the distinction between functions and data is blurry) and uses it to create a new function.

To use Function Builder, we write a higher-order function that returns a function. The Function Builder implementation encodes some pattern we’ve discovered.

For example, to create a consonant? predicate from a vowel? predicate, we create a new function that calls vowel? and negates the result. To create odd? from even?, we create a function that calls even? and negates the result. To create dead? from alive?, we create a function that calls dead? and negates the result.

There’s an obvious pattern here. We can encode it with a Function Builder implementation named negate. The negate function takes in a function and returns a new one that calls the passed-in function and negates the result.

Another common use for Function Builder is when we’ve got a piece of static data we need to use as the basis for some action. For instance, we could convert a static percentage to a function that calculates percentages by writing a function that takes in the percentage and returns a function of one argument. This function takes in a number to calculate a percentage of and uses the percentage stored in its closure to do so. We’ll see several examples of both flavors of Function Builder a bit later on.

Code Sample: Functions from Static Data

One way to use Function Builder is to create functions out of static data. This lets us take a bit of data—a noun—and turn it into an action—a verb. Let’s look at a couple of examples, starting with a function that takes a percentage and creates a function that calculates discounted prices based on those percentages.

Discount Calculator Builder

The Function Builder discount takes in a percentage between 0 and 100 and returns a function that computes a discounted price based on that percentage. Passing 50 into discount returns a function that calculates a 50 percent discount, 25 gets us a 25 percent discount, and so on. Let’s take a look at the Scala version.

In Scala

Our Scala code defines discount, which takes a Double, named percent, and checks to ensure that it’s between 0 and 100. It then creates a function that uses discountPercentage to calculate a discount. Here’s the code:

Let’s take a look at how it works. The simplest way to use discountedPrice is to have it create an anonymous function, which we call directly. Here we use it to calculate a 50 percent discount on a price of 200:

And here we use it to calculate a 0 percent discount (full price) and a 100 percent discount (free!), respectively:

If we need to use the discount function more than once, we can name it. Here we do so and use it to calculate discounted totals on a couple of vectors of items:

In Clojure

This example works much the same in Clojure. The only interesting difference is that we can use Clojure’s preconditions to ensure that the discount is in the valid range. Let’s take a look at the code:

We can create a discounted price and call it as an anonymous function:

As advertised, trying to create a discount outside the acceptable range throws an exception:

And if we want to name our discount function to use it multiple times, we can do so:

The discount calculator is a fairly simple example; we’ll take a look at one that’s a bit more involved in the next section.

Map Key Selector

Let’s take a look at a more involved implementation of Function Builder. The problem we’re trying to solve is this: we’ve got a data structure consisting of maps nested inside each other, and we want to create functions that help us pick out values, possibly from deeply nested parts.

In a way, this is writing a very simple declarative language to pick values out of deeply nested maps. This is a lot like how XPath lets us select an arbitrary element from a deeply nested XML structure, or how a CSS selector lets us do the same with HTML.

Our solution starts with creating a function, selector, which takes a path to the data we’re looking for. For instance, if we’ve got a map that represents a person, which contains a name key whose value is another map, which contains a first key whose value is the first name, we want to be able to create a selector for the first name like this: selector(’name, ’first). We can see this in the code below:

This sort of structure is extremely handy when working with structured data like XML or JSON. The data can be parsed into a nested structure, and this type of Function Builder can help pick it apart.

In Scala

The Scala version of selector creates functions that can pick values out of deeply nested maps, as described previously. The selectors that it creates will return Some(Any) if it can find the nested value; otherwise it returns None.

To create a selector, we need to pass in several Symbols corresponding to the keys in the path we want to select. Since this is all we need to pass into selector, we can use Scala’s support for varargs instead of passing in an explicit list; this means that creating a selector to pick a street name from a person’s address looks like this:

Once created, a map is passed into the selector, and it attempts to select a value based on the path it was given when it was created by recursively walking through the map. This is a slightly tricky bit of code, so let’s look at the whole thing and then break it down into smaller parts:

Let’s start by examining the signature of selector:

This says that selector takes a variable number of Symbol arguments and returns a function. The function it returns itself takes a map from Symbol to Any and returns an Option[Any].

The first line simply checks to make sure that the path has at least one element and throws an exception if it doesn’t:

The meat of the function is a nested, recursive helper function. Let’s take a look at its type signature:

This says that selectorHelper takes a sequence of Symbols as a path and a data structure that consists of a map from Symbol to Any. It returns an Option[Any], which represents the final value we’re trying to find with the selector. In the above example, this would be the name of a person’s street.

Next, we get the base case for our recursion. This happens when we reach the end of the path. We find the value we’re looking for and return it. The get method returns None if the value doesn’t exist:

The largest piece of code contains the tail recursive call. Here, we get the current piece of the data structure. If it exists, then we call the helper function recursively with the remainder of the path and the data structure we just picked out. If it doesn’t exist, or if it doesn’t have the proper type, we return None:

Finally, here is the last line, which just returns a function that calls selectorHelper with the appropriate arguments:

Let’s take a closer look at how we can use selector, starting with a very simple example, a map that has a single key-value pair:

Of course the real power is only apparent when we start working with nested data structures, like so:

If the selector doesn’t match anything, a None is returned:

In Clojure

The Clojure version of selector is much simpler than the Scala one. In part, this is because Clojure is dynamically typed, so we don’t have to worry about the type system as we did in Scala. In addition, Clojure has a handy function called get-in, which is tailor-made to pick values out of deeply nested maps.

Let’s take a quick look at get-in before we dig into the code. The get-in function takes a nested map as its first argument and a sequence that represents the path to the value you’re looking for. Here’s an example of using it to pick a street name from a nested map:

Building selector on top of get-in is extremely straightforward. We’ve just got to add a validator to ensure that the path isn’t empty and use varargs for the path. Here’s the code:

Using it is just as easy as the Scala version. Here we pick out a person name from a flat map:

And here we pick out a street name from a more deeply nested one:

Before we move on, here’s a quick note on the relative complexity of the Scala and Clojure versions of this example. The fact that Clojure has get-in, which does almost exactly what we want to do, helps make the Clojure version much more concise. The other factor is that Clojure is a dynamically typed language. Since the nested maps can hold values of any type, this takes some type system gymnastics to handle in Scala, which is statically typed.

In Clojure, using maps to hold data like this is very idiomatic. In Scala, it’s more common to use classes or case classes. However, for this sort of very dynamic problem, I much prefer just keeping things in a map. Using a map means we can manipulate the data structure with all the built-in tools for manipulation maps and collections.

Functions from Other Functions

Since functions in the functional world are themselves pieces of data that can be manipulated, it’s common to use Function Builder to transform one function into another. This can be done very simply by just creating a new function that manipulates the return value of another function. For instance, if we have a function isVowel and we want a function isNotVowel, we can simply have it delegate to isVowel and negate the result. This creates a complementary function, as the Scala code shows:

In this example, we’ll take a closer look at two other ways to create functions from existing functions: function composition and partial function application. Function composition lets us take multiple functions and chain them together. Partial function application lets us take one function and some of its arguments and create a new function of fewer arguments. These are two of the most generally useful ways of creating functions from functions.

Function Composition

Function composition is a way to chain function invocations together. Composing a list of functions together gives us a new function that invokes the first function, passes its output to the next, which passes it to the next, and so on until a result is returned.

In many ways, function composition is similar to the way that Pattern 9, ​Replacing Decorator​, is used. With the Decorator pattern, multiple decorators, each of which does one part of some task, are chained together. Here multiple functions are chained together.

It’s possible to use function composition by simply chaining together functions by hand, but since this is such a common task, functional languages provide first class support for it. Clojure and Scala are no exception here, so let’s take a look at it.

In Scala

In Scala, we can compose functions together with the compose operator. As a simple example, let’s define three functions, appendA, appendB, and appendC, which append the strings "a", "b", and "c", respectively, as the code shows:

Now if we want a function that appends all three letters, we can define it like so using function composition:

As the name suggests, this appends the letters c, b, and a, in that order. It’s equivalent to writing a function that takes an argument, passes it into appendC, takes the returned value and passes it into appendB, and finally passes that returned value into appendA:

This is a trivial example, but it illustrates an important thing about function composition, which is the order in which the composed functions are called. The last function in the composition chain is called first, and the first function is called last, which is why c is the first letter appended to our string.

Let’s take a look at a more involved example. One common situation that comes up in web application frameworks is the need to pass an HTTP request through a series of user-defined chunks of code. J2EE’s servlet filters,^[7] which pass a request through a chain of filters before it is handled, are a common example of such a filter chain.

Filter chains allow application code to do anything that needs to be done before request handling, like decrypting and decompressing the request, checking authentication credentials, logging to a request log, and so forth. Let’s sketch out how we’d do this using function composition. First, we’ll need a way to represent HTTP requests. For the purpose of this example, we’ll keep it simple and stick to a map of request headers and a string request body:

Next, let’s define some filters. Each filter is a function that takes in an HttpRequest, does something, and returns an HttpRequest. For this simple example, we’re returning the same HttpRequest; but if the filter needed to modify or add something to the request, it could do so by creating a new HttpRequest with its modifications.

Here are a couple of example filters—the first mimics checking an Authorization header and adding a user principal to the request if it’s valid, and the second mimics logging out a request fingerprint for troubleshooting:

Finally, we need a function that takes a sequence of filters and composes them together. We can do this by simply reducing the composition function over the sequence:

Let’s watch it work by composing the sample filters into a single filter chain and running a test HttpRequest through it:

As we can see, the filter chain properly runs the HttpRequest through each filter in the chain, which adds a user principal to the request and logs our fingerprint to the console.

In Clojure

The easiest way to do function composition in Clojure is to use comp. Here we are using it to compose together the string appenders:

This works much like the Scala version:

In Clojure we’ll model the HTTP request itself, as well as the headers, as a map. A sample request looks like so:

Our sample filter functions pick keys out of a map and use nil instead of None to represent missing values. Here they are, along with the function builder, compose-filters, to compose them into a filter chain:

And here’s that filter chain in action, running through the filters, performing them, and finally returning the HTTP request:

Function composition is a very general operation, and we’ve only touched on a few uses of it here. Any time you find yourself calling the same set of functions in the same order multiple times, or you have a dynamically generated list of functions that need to be chained together, function composition is a good place to turn to.

Partially Applied Functions

While function composition takes multiple functions and chains them together, partially applying a function takes one function and a subset of the arguments that that function takes and returns a new function. The new function has fewer arguments than the original and keeps track of the subset that was passed in when the partially applied function was created so it can use them later when it gets the rest of the arguments.

Let’s see how it works in Scala.

In Scala

Partial function application is another functional feature that’s important enough to warrant first-class support in Scala. The way it works is that you call a function and replace the arguments you don’t currently have values for with underscores. For example, if we’ve got a function that adds two integers together and we want a function that adds 42 to a single integer, we could create it like this:

As the code below shows, addFortyTwo is a function of one argument, to which it adds 42.

Creating partially applied functions is simple, but spotting when to use them can be a bit tough. Here’s one example where they come in handy. Say we’ve got a function that calculates income tax by state, and we want to create functions that let us calculate the income tax for a particular state. We can use a partially applied function to do it, like so:

This correctly calculates taxes for the different states:

In Clojure

Partially applying functions in Clojure is similar to how it’s done in Scala, but there is one twist. To keep its syntax simple, Clojure only allows for the arguments that the function is being partially applied to, to come at the start of the argument list. For example, we could still write add-forty-two, much as we did in Scala, as this code shows:

But to write ny-tax and pa-tax, we’d have to swap the arguments to tax-for-state around, like this:

Partially applied functions are very simple to use, but I often find it a bit tricky to know when to use them. I usually catch myself calling the same function over and over again, with a subset of the arguments remaining the same. Then a light bulb goes off and I realize I can clean that up a bit by using a partially applied function.

Discussion

In this section, we’ve covered some of the more general ways to use Function Builder, but these are by no means the only ways. The Clojure and Scala libraries contain many other examples, since this is an extremely common pattern in the functional world.

While most of the Clojure and Scala examples were very similar, the examples in ​Map Key Selector​, differed drastically. In particular, the Scala version was much more verbose. In part, this is because Clojure has an extremely handy get-in function that does almost exactly what we need; however, a large part of the difference was caused by Scala’s type system.

Since Scala is statically typed, we had to specify types for the contents of the maps that we dealt with. Internal nodes were themselves Maps, while leaf nodes could be anything at all. This led to the slight bit of type system gymnastics we had to do in the Scala version.

This is a general trade-off between dynamic and static typing. Even with a powerful type system like Scala’s, there’s still a cost to static typing in terms of the complexity it can add and in just understanding how the type system works. The trade-off is that we can catch many errors at compile time that would otherwise become runtime errors with a dynamic type system.

Related Patterns

Pattern 1, ​Replacing Functional Interface​

Pattern 9, ​Replacing Decorator​