Funcs in Enumerables

I’ve seen plenty of examples of Funcs as parameters to functions, but I’m not sure many developers realize that we can put them in an enumerable and create some interesting behaviors.

First is the obvious one—put them in an array to act on the same data multiple times:

private IEnumerable<Func<Employee, string>> descriptors = new []
{
    x => "First Name = " + x.firstName,
    x => "Last Name = " + x.lastName,
    x => "MiddleNames = string.Join(" ", x.MiddleNames)
}

public string DescribeEmployee(Employee emp) =>
   string.Join(Environment.NewLine, descriptors.Select(x => x(emp)));

Using this technique, we can have a single original source of data (here, an Employee object) and have multiple records of the same type generated from it. In this case, we aggregate using the built-in .NET method string.Join to present a single, unified string to the end user.

This approach has a few advantages over a simple StringBuilder. First, the array can be assembled dynamically. We could have multiple rules for each property and how it’s rendered, which could be selected from a set of local variables depending on custom logic.

Second, this is an enumerable, so by defining it this way, we’re taking advantage of a feature of enumerables called lazy evaluation (introduced in Chapter 2). The thing about enumerables is that they aren’t arrays; they aren’t even data. They’re just pointers to something that will tell us how to extract the data. It might well be—and, in fact, usually is the case—that the source behind the enumerable is a simple array, but not necessarily. An enumerable requires a function to be executed each time the next item is accessed via a foreach loop. Enumerables were developed to transform into actual data only at the very last possible moment—typically, when starting a foreach loop iteration. Most of the time, this doesn’t matter if an array held in-memory somewhere is feeding the enumerable, but if an expensive function or lookup to an external system is powering it, lazy loading can be incredibly useful to prevent unnecessary work.

The elements of an enumerable are evaluated one at a time and only when their turn has come to be used by whatever process is performing the enumeration. For example, if we use the LINQ Any function to evaluate each element in an enumerable, Any will stop enumerating the first time an element is found that matches the specified criteria, meaning the remaining elements will be left unevaluated.

Lastly, from a maintenance perspective, this technique is easier to live with. Adding a new line to the final result is as easy as adding a new element to the array. This approach also acts as a restraint to future programmers, making it harder for them to try to put too much complex logic where it doesn’t belong.

A Super-Simple Validator

Let’s imagine a quick validation function, which typically looks like this:

public bool IsPasswordValid(string password)
{
    if(password.Length <= 6)
        return false;

    if(password.Length > 20)
        return false;

    if(!password.Any(x => Char.IsLower(x)))
        return false;

    if(!password.Any(x => Char.IsUpper(x)))
        return false;

    if(!password.Any(x => Char.IsSymbol(x)))
        return false;

    if(password.Contains("Justin", StringComparison.OrdinalIgnoreCase)
        && password.Contains("Bieber", StringComparison.OrdinalIgnoreCase))
            return false;

    return true;
}

Well, for a start, that’s a lot of code for what is, in fact, a fairly simple set of rules. The imperative approach forces you to write a whole heap of repetitive boilerplate code. On top of that, if we want to add in another rule, that’s potentially around four new lines of code to add when really only one is especially interesting to us.

If only there were a way to compact this code into just a few simple lines. Well, since you asked so nicely, here you go:

public bool IsPasswordValid(string password) =>
    new Func<string, bool>[]
    {
        x => x.Length > 6,
        x => x.Length <= 20,
        x => x.Any(y => Char.IsLower(y)),
        x => x.Any(y => Char.IsUpper(y)),
        x => x.Any(y => Char.IsSymbol(y)),
        x => !x.Contains("Justin", StringComparison.OrdinalIgnoreCase)
            && !x.Contains("Bieber", StringComparison.OrdinalIgnoreCase)
    }.All(f => f(password));

Not so long now, is it? What have we done here? We’ve put all the rules into an array of Funcs that turn a string into a bool—i.e., check a single validation rule. We use a LINQ statement: .All(). The purpose of this function is to evaluate whatever lambda expression we give it against all elements of the array it’s attached to. If a single one of these returns false, the process is terminated early, and false is returned from All() (as mentioned earlier, the subsequent values aren’t accessed, so lazy evaluation saves us time by not evaluating them). If every single one of the items returns true, All() also returns true.

We’ve effectively re-created the first code sample, but the boilerplate code we were forced to write—if statements and early returns—is now implicit in the structure.

This also has the advantage of once again being easy to maintain as a code structure. If we wanted, we could even generalize it into an extension method. I do this often:

public static bool IsValid<T>(this T @this, params Func<T,bool>[] rules) =>
    rules.All(x => x(@this));

This reduces the size of the password validator yet further and gives us a handy, generic structure to use elsewhere:

public bool IsPasswordValid(string password) =>
    password.IsValid(
        x => x.Length > 6,
        x => x.Length <= 20,
        x => x.Any(y => Char.IsLower(y)),
        x => x.Any(y => Char.IsUpper(y)),
        x => x.Any(y => Char.IsSymbol(y)),
        x => !x.Contains("Justin", StringComparison.OrdinalIgnoreCase)
            && !x.Contains("Bieber", StringComparison.OrdinalIgnoreCase)
    )

At this point, I hope you’re reconsidering ever writing something as long and ungainly as that first validation code sample ever again.

I think an IsValid check is easier to read and maintain, but if we want a piece of code that is much more in line with the original code sample, we can create a new extension method by using Any() instead of All():

public static bool IsInvalid<T>(
    this T @this,
    params Func<string,bool>[] rules) =>

This means that the Boolean logic of each array element can be reversed, as it was originally:

public bool IsPasswordValid(string password) =>
    !password.IsInvalid(
        x => x.Length <= 6,
        x => x.Length > 20,
        x => !x.Any(y => Char.IsLower(y)),
        x => !x.Any(y => Char.IsUpper(y)),
        x => !x.Any(y => Char.IsSymbol(y)),
        x => x.Contains("Justin", StringComparison.OrdinalIgnoreCase)
            && x.Contains("Bieber", StringComparison.OrdinalIgnoreCase)
    )

If we want to maintain both functions, IsValid() and IsInvalid(), because each has its place in our codebase, it’s probably worth saving some coding effort and preventing a potential maintenance task in the future by simply referencing one in the other:

public static bool IsValid<T>(this T @this, params Func<T,bool>[] rules) =>
    rules.All(x => x(@this));

public static bool IsInvalid<T>(this T @this, params Func<T,bool>[] rules) =>
    !@this.IsValid(rules);

Use it wisely, my young functional Padawan learner.

Pattern Matching for Old Versions of C#

Pattern matching is one of the best features of C# in recent years, along with record types, but it isn’t available in anything except the most recent .NET versions. (See Chapter 3 for more details on native pattern matching in C# 7 and up.)

Is there a way to allow pattern matching to happen, but without needing up upgrade to a newer version of C#? There most certainly is. It is nowhere near as elegant as the native syntax in C# 8, but it provides a few of the same benefits.

In this example, we’ll calculate the amount of tax someone should pay based on a grossly simplified version of the UK income tax rules. Note that these really are much simpler than the real thing. I don’t want us to get too bogged down in the complexities of taxes.

The rules to apply look like this:

• If yearly income is less than or equal to £12,570, no tax is taken.

• If yearly income is between £12,571 and £50,270, take 20% tax.

• If yearly income is between £50,271 and £150,000, take 40% tax.

• If yearly income is over £150,000, take 45% tax.

If we wanted to write this longhand (nonfunctionally), it would look like this:

decimal ApplyTax(decimal income)
{
    if (income <= 12570)
        return income;
    else if (income <=50270)
        return income * 0.8M;
    else if (income <= 150000)
        return income * 0.6M;
    else
        return income * 0.55M;
}

Now, in C# 8 and onward, switch expressions would compress this to a few lines. So long as we’re running at least C# 7 (.NET Framework 4.7), this is the style of pattern matching we can create:

var inputValue = 25000M;
var updatedValue = inputValue.Match(
    (x => x <= 12570, x => x),
    (x => x <= 50270, x => x * 0.8M),
    (x => x <= 150000, x => x * 0.6M)
).DefaultMatch(x => x * 0.55M);

We’re passing in an array of tuples containing two lambda expressions. The first determines whether the input matches against the current pattern; the second is the transformation in value that occurs if the pattern is a match. There’s a final check to see whether the default pattern should be applied—i.e., because none of the other patterns were a match.

Despite being a fraction of the length of the original code sample, this contains all the same functionality. The matching patterns on the lefthand side of the tuple are simple, but they can contain expressions as complicated as we’d like and could even be calls to whole functions containing detailed criteria to match on.

So, how do we make this work? This is an extremely simple version that provides most of the functionality required:

public static class ExtensionMethods
{
    public static TOutput Match<TInput, TOutput>(
        this TInput @this,
        params (Func<TInput, bool> IsMatch,
        Func<TInput, TOutput> Transform)[] matches)
    {
        var match = matches.FirstOrDefault(x => x.IsMatch(@this));
        var returnValue = match?.Transform(@this) ?? default;
        return returnValue;
    }
}

We use the LINQ method FirstOrDefault() to first iterate through the lefthand functions to find one that returns true (i.e., one with the right criteria), and then call the righthand conversion Func to get the modified value.

This is fine, except that if none of the patterns match, we’ll be in a bit of a fix. Most likely, we’ll have a null reference exception.

To cover this, we must force the need to provide a default match (the equivalent of a simple else statement, or the _ pattern match in switch expressions). The answer is to have the Match function return a placeholder object that either holds a transformed value from the Match expressions or executes the Default pattern lambda expression. The improved version looks like this:

public static MatchValueOrDefault<TInput, TOutput> Match<TInput, TOutput>(
  this TInput @this,
  params (Func<TInput, bool>,
  Func<TInput, TOutput>)[] predicates)
{
    var match = predicates.FirstOrDefault(x => x.Item1(@this));
    var returnValue = match?.Item2(@this);
    return new MatchValueOrDefault<TInput, TOutput>(returnValue, @this);
}

public class MatchValueOrDefault<TInput, TOutput>
{
    private readonly TOutput value;
    private readonly TInput originalValue;

    public MatchValueOrDefault(TOutput value, TInput originalValue)
    {
        this.value = value;
        this.originalValue = originalValue;
    }
}

public TOutput DefaultMatch(Func<TInput, TOutput> defaultMatch)
{
    if (EqualityComparer<TOutput>.Default.Equals(default, this.value))
    {
        return defaultMatch(this.originalValue);
    }
    else
    {
        return this.value;
    }
}

This approach is severely limited compared to what can be accomplished in the latest versions of C#. No object type matching occurs, and the syntax isn’t as elegant, but it’s still usable and could save an awful lot of boilerplate as well as encourage good code standards.

In versions of C# that are older still and that don’t include tuples, we can consider the use of KeyValuePair<T,T>, though the syntax is far from attractive. What, you don’t want to take my word? OK, here we go. Don’t say I didn’t warn you…​

The Extension() method itself is about the same and just needs a small alteration to use KeyValuePair instead of tuples:

public static MatchValueOrDefault<TInput, TOutput> Match<TInput, TOutput>(
  this TInput @this,
  params KeyValuePair<Func<TInput, bool>, Func<TInput, TOutput>>[] predicates)
{
    var match = predicates.FirstOrDefault(x => x.Key(@this));
    var returnValue = match.Value(@this);
    return new MatchValueOrDefault<TInput, TOutput>(returnValue, @this);
}

And here’s the ugly bit. The syntax for creating KeyValuePair objects is pretty awful:

var inputValue = 25000M;
var updatedValue = inputValue.Match(
    new KeyValuePair<Func<decimal, bool>, Func<decimal, decimal>>(
        x => x <= 12570, x => x),
    new KeyValuePair<Func<decimal, bool>, Func<decimal, decimal>>(
        x => x <= 50270, x => x * 0.8M),
    new KeyValuePair<Func<decimal, bool>, Func<decimal, decimal>>(
        x => x <= 150000, x => x * 0.6M)
).DefaultMatch(x => x * 0.55M);

So we can still have a form of pattern matching in C# 4, but I’m not sure how much we’re gaining by doing it. That’s perhaps up to you to decide. At least I’ve shown you the way.

Query Adjacent Elements

Let’s start with a relatively simple example. Imagine that we want to run through an enumerable of integers, to see whether it contains any numbers that are consecutive. An imperative solution would likely look like this:

public IEnumerable<int> GenerateRandomNumbers()
{
    var rnd = new Random();
    var returnValue = new List<int>();
    for (var i = 0; i < 100; i++)
    {
        returnValue.Add(rnd.Next(1, 100));
    }
    return returnValue;
}

public bool ContainsConsecutiveNumbers(IEnumerable<int> data)
{
    // OK, you caught me out: OrderBy isn't strictly imperative, but
    // there's no way I'm going to write out a sorting algorithm out
    // here just to prove a point!
    var sortedData = data.OrderBy(x => x).ToArray();

    for (var i = 0; i < sortedData.Length - 1; i++)
    {
        if ((sortedData[i] + 1) == sortedData[i + 1])
            return true;
    }

    return false;
}

var result = ContainsConsecutiveNumbers(GenerateRandomNumbers());
Console.WriteLine(result);

To make this code functional, as is often the case, we need an extension method. This would take the enumerable, extract its enumerator, and control the customized behavior.

To avoid use of an imperative-style loop, we’ll use recursion here. Recursion (introduced in Chapters 1 and 2) is a way of implementing an indefinite loop by having a function call itself repeatedly.²

I’ll revisit the concept of recursion in Chapter 9. For now, let’s use the standard, simple version of recursion:

public static bool Any<T>(this IEnumerable<T> @this, Func<T, T, bool> evaluator)
{
    using var enumerator = @this.GetEnumerator();
    var hasElements = enumerator.MoveNext();
    return hasElements && Any(enumerator, evaluator, enumerator.Current);
}

private static bool Any<T>(IEnumerator<T> enumerator,
        Func<T, T, bool> evaluator,
        T previousElement)
{
    var moreItems = enumerator.MoveNext();
    return moreItems && (evaluator(previousElement, enumerator.Current)
        ? true
        : Any(enumerator, evaluator, enumerator.Current));

}

So, what’s happening here? This approach is kind of like juggling, in a way. We start by extracting the enumerator and moving to the first item.

Inside the private function, we accept the enumerator (now pointing to the first item), the “are we done” evaluator function, and a copy of that same first item.

Then we immediately move to the next item and run the evaluator function, passing in the first item and the new Current, so they can be compared.

At this point, either we find out we’ve run out of items or the evaluator returns true, in which case we can terminate the iteration. If MoveNext() returns true, we check if the previousValue and Current match our requirement (as specified by evaluator). If they do, we finish and return true; otherwise, we make a recursive call to check the rest of the values.

This is the updated version of the code to find consecutive numbers:

public IEnumerable<int> GenerateRandomNumbers()
{
    var rnd = new Random();

    var returnValue = Enumerable.Repeat(0, 100)
        .Select(x => rnd.Next(1, 100));
    return returnValue;
}

public bool ContainsConsecutiveNumbers(IEnumerable<int> data)
{
    var sortedData = data.OrderBy(x => x).ToArray();
    var result = sortedData.Any((prev, curr) => cur == prev + 1);
    return result;
}

It would also be easy enough to create an All() method based on the same logic, like so:

public static bool All<T>(
    this IEnumerator<T> enumerator,
    Func<T,T,bool> evaluator,
    T previousElement)
{
    var moreItems = enumerator.MoveNext();
    return moreItems
        ? evaluator(previousElement, enumerator.Current)
            ? All(enumerator, evaluator, enumerator.Current)
            : false
        : true;
}

public static bool All<T>(this IEnumerable<T> @this, Func<T,T,bool> evaluator)
{
    using var enumerator = @this.GetEnumerator();
    var hasElements = enumerator.MoveNext();
    return hasElements
        ? All(enumerator, evaluator, enumerator.Current)
        : true;
}

The only differences between All() and Any() are the conditions for deciding whether to continue and whether you need to return early. With All(), the point is to check every pair of values and return out of the loop early only if one is found not to meet the criteria.

Iterate Until a Condition Is Met

The technique described in this section is basically a replacement for a while loop, so there’s another statement we don’t necessarily need.

For this example, let’s imagine what the turn system might be like for a text-based adventure game. For younger readers, this is what we had in the old days, before graphics. You used to have to write what you wanted to do, and the game would write what happened—kind of like a book, except you wrote what happened yourself.

The basic structure of one of those games was something like this:

1. Write a description of the current location.

2. Receive user input.

3. Execute the requested command.

Here’s how imperative code might handle that situation:

var gameState = new State
{
    IsAlive = true,
    HitPoints = 100
};

while(gameState.IsAlive)
{
    var message = this.ComposeMessageToUser(gameState);
    var userInput = this.InteractWithUser(message);
    this.UpdateState(gameState, userInput);

    if(gameState.HitPoints <= 0)
        gameState.IsAlive = false;
}

In principle, what we want is a LINQ-style Aggregate() function, but one that doesn’t loop through all the elements of an array and then finish. Instead, we want the function to loop continuously until our end condition is met (the player is dead). I’m simplifying a little here (obviously, our player in a proper game could win as well). But my example game is like life, and life’s not fair!

The extension method for this is another place that would benefit from tail-recursion optimized calls, and I’ll be presenting options for that in Chapter 9. For now, though, we’ll just use simple recursion (which may become an issue if the game has a lot of turns) to avoid introducing too many ideas too soon:

public static class ExtensionMethods
{
    public static T AggregateUntil<T>(
      this T @this,
      Func<T,bool> endCondition,
      Func<T,T> update) =>
        endCondition(@this)
             ? @this
             : AggregateUntil(update(@this), endCondition, update);
}

Using this, we can do away with the while loop entirely and transform the entire turn sequence into a single function, like so:

var gameState = new State
{
    IsAlive = true,
    HitPoints = 100
};

var endState = gameState.AggregateUntil(
    x => x.HitPoints <= 0,
    x => {
        var message = this.ComposeMessageToUser(x);
        var userInput = this.InteractWithUser(message);
        return this.UpdateState(x, userInput);
    });

This isn’t perfect but it’s functional now. There are far better ways of handling the multiple steps to update the game’s state, and the issue of how to handle user interaction in a functional manner remains too. Chapter 13 covers those topics.