There are several generic containers such as List<T>, Dictionary<Tkey,Tvalue>, and so on, which you have learnt about in the previous chapters. Now, let's take a look at some of the patterns involving these generic containers that show up more often in code.

Since Tuple is a new inclusion in the .NET 4.0 Framework and can prove to be handy in a lot of situations, I thought it would be a great place to showcase an unusual use of this beautiful data representation technique.

Let's say we have situations involving four integers. This is generally represented using nested if-else blocks, as shown in the following code. However, we can use a Tuple to refactor this code, as we shall see in a while:

Follow the given steps:

y <= x and z > y and w > 10

Tuple<bool, bool, bool, bool, Func<int, int, int, int, string>>

Func<int, int, int, int, string> function = matchingRule.Item5;

The following are the best practices to be followed while using Generics:

Not all generic containers are geared to do each job well, as we already know it. Here is a table that will let you pick a generic container that supports some features listed in the left-hand column.

You can download the GenGuru app from the book's website. It's a kind of wizard that will recommend the generic containers available in .NET Generics that you should use. It will ask you few questions and recommend a collection. You can see it in action at http://sudipta.posterous.com/private/niixaeFlmk.

The following are the best practices while creating custom generic collections:

Remember these points while coming up with your own generic collection.

Let's accept that we are impatient. Rather, we are becoming more and more impatient with each year. Part of this is due to the availability of processors that are a zillion times faster than the previous ones and come in pairs in almost every computer. Buying a single core machine these days is almost impossible. Computer hardware is faster than ever. Software developers have to put in extra effort to put these new powers to good use.

"Algorithms + Data Structures = Programs" said Niklaus Wirth. He was absolutely right in the title. If you add the word 'faster' to all the words in the equation mentioned earlier, it would result in "Faster Algorithms + Faster Data Structures = Faster Programs".

For a moment, forget whatever you know about data structures and think like a mathematician rather than a programmer. You should see that there are basically only a few types of data structures possible. They are Lists, Associative Containers, Sets, and sorted versions of each of these. It doesn't matter to a client code developer whether you use "Red-black tree" or "HashTable" internally to implement your dictionary as long as they both expose the same functionality. Take an ATM machine, for example. As long as the machine functions in a way people are familiar with, no one complains. But users complain if the machine requires some extra input or the interface changes.

In this chapter, we shall try to see which generic container outperforms its peers in practice even if their theoretical worst-case time complexity limit is identical. For example, HashSet<T> in System.Collections.Generics namespace and HashSet<T> in C5 collections, both are documented to offer constant time lookup activity. But in practice, as you will find in a little while, HashSet<T> in System.Collections.Generics is way faster than HashSet<T> in C5 collections.

We will investigate the following important situations for each kind of data structures:

We shall put all data structures under a lot of stress! We shall perform the same operations over and over again starting at 100,000 times to 2,000,000 times with a step size of 100,000 operations at a time. I have put the results in spreadsheets and have generated graphs and tables for your future reference. All source code used in these benchmarking experiements are also available for download from the book's website. Benchmarking results are recorded in tables. In this chapter, we shall use fewer rows of these tables to save some space. If you want the entire table, you can always get that from the spreadsheet.

Codes used to benchmark these data structures are given right next to the benchmarking result. Actual results (measurement figures) that you would get from running these code snippets on your system could be very different in absolute terms; however, they will surely share the same relative resemblance.

In the following tables, the figures that represent the time taken for the operation are measured in milliseconds. Create a console application and put the code and assemblies required (for example, C5.dll when you are using generic collections from this namespace) and the benchmarking code in the Main() method. You should be good to go. Further, you can download the Excel file from the book's website if you want to add some more values and see it in reference with others. Excel also enables you to add trend lines. So you can experiment with that too. The advice is don't trust documentation. Don't trust me. Do the profiling yourself. You as the developer of a program know what your requirement is better than anyone else. Use the strategies discussed in this chapter to find out the collection that works best for you. Good Luck!

How much time does it take to check whether an element is present in a list-based collection?

In order to determine whether a given item is in a collection or not, we have to call the Contains() method on linear containers.

The following are the steps to create the benchmarking code that I have used. You shall have to change the Contain() method accordingly, to alter it for the next benchmarking on another equivalent container:

What we just did, can hardly be termed as benchmarking. We have to compare the results obtained for HashSet<T> with other equivalent containers such as SortedSet<T>, C5.HashSet<T>, and so on. In order to do that, we have to run the previous program after changing it to use the container we want. For example, if we want to benchmark C5.HashSet<T> next, we'll have to perform the following steps:

This is the way I got the performance figures of all linear lists such as containers by slightly changing the given benchmarking code. Once I got these numbers, I kept saving them on an Excel sheet.

Towards the end, I got something similar to the following screenshot:

From these data, I generated the following graph:

I was trying to check whether a given element is there in the list or not. This is also known as checking for membership in a given container. Thus the title of the graph is "Performance of Membership Checking".

You can download the Excel sheet with all these figures from the book's website.

It will be a great idea to create a tag cloud as a performance poster. From this data, I created the following tag cloud using Wordle (www.wordle.net):

I am quite happy with the end result of my benchmarking. This will stick longer in my brain than the actual figures and the graph. This means HashSet<T> is the fastest of all containers when it comes to membership checking. As a rule of thumb, anything that you can't read without a magnifying glass (such as LinkedList shown above the letter e in HashSet) should be among your last choices for this functionality.

If you are interested to know how I created this tag cloud with container names, please refer to the article at http://sudipta.posterous.com/private/kzccCdJaDg.

You can download benchmarking code for all these containers for membership checking from http://sudipta.posterous.com/private/HmrocplDxg.

I shall use the same pattern for all other benchmarking experiments in this chapter. I reckoned that it might be difficult to follow for some of you, so I created a screencast of the first experiment from start to finish. You can download this screencast (PerformanceAnalysis_Screencast.MP4) from the book's website.

How much time does it take to find the index of the first occurrence of an item in a list such as collection?

When dealing with a list such as containers, it is very common to face a situation where we need to know the index of a given element in the collection. Here, we shall compare the following containers to see how efficient they are in this situation:

List<T>
C5.LinkedList<T>
C5.ArrayList<T>
C5.HashedArrayList<T>
C5.HashedLinkedList<T>
C5.TreeSet<T>

The approach remains the same as the previous task. The benchmarking code is as follows:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace PerformanceMap
{
  class Program
  {
    static void Main(string[] args)
    {
      List<char> codes = new List<char>();

      string alphabets = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
      foreach (char c in alphabets)
      codes.Add(c);

      Dictionary<int, double> perfMap = 
      new Dictionary<int, double>();

      int N = 100000;

      int index = 0; 


      for (; N <= 2000000; N += 100000)
      {
        DateTime start = DateTime.Now;
        for (int k = 0; k < N; k++)
        {
          index = codes.IndexOf('Z'),
        }
        DateTime end = DateTime.Now;
        perfMap.Add(N, end.Subtract(start).TotalMilliseconds);
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]);
      Console.ReadLine();            
    }
  }
}

As you run the code, you shall get an output similar to the following screenshot:

And using the previous approach and running the program for all other containers mentioned, we get the following output in an Excel sheet:

After plotting the entire data, I got the following graph:

From this graph, it is evident that C5.HashedArrayList<T> is the fastest when it comes to finding the first index of occurrence of a given element in the container.

Following the previous approach, I got the following tag cloud:

Also, from the previous graph, it is evident that C5.HashedArrayList<T> is the fastest and C5.HashedLinkedList<T> is prohibitively slow. So, think twice before you call IndexOf() on a linked implementation.

To modify the benchmarking code for all other containers, just change the following line with the appropriate instantiation call:

List<char> codes = new List<char>(); 

For example, if you want to benchmark C5.HashedArrayList<T>, then follow the given steps:

How much time does it take to find the last index of an item in a list-based collection?

If you go by your intuition, it might seem obvious that the container that offers the lowest time for first index lookup of a given element will do so for the last index of occurrence too. However, soon you shall find that's not the case. Here, we shall benchmark the same containers as in the previous one.

The benchmarking code will be exactly the same. Except, instead of the IndexOf() method, you have to call the LastIndexOf() method on the containers.

The following is the result for all these containers:

The graph depicting the relative performance of all these containers for this operation is as follows:

And the following is what I got putting the data to Wordle, as I did earlier:

From the previous graph, it is pretty much evident that C5.LinkedList<T>, C5.ArrayList<T>, and List<T> offer equivalent performance. Anything beyond these collections would be an underperformer for this task.

How much time it takes to insert a single element in a random location in a list-based collection?

Sometimes we need to insert an element at a random location. Think about sorting, for example. Not all collections are geared to make this efficient. However, I shall benchmark the containers that support this operation:

List<T>
LinkedList<T>
C5.HashedLinkedList<T>
C5.HashedArrayList<T>
C5.LinkedList<T>
C5.ArrayList<T>

Apart from LinkedList<T> in the System.Collections.Generic namespace, all other containers listed previously have a method called insert() to insert an element at the mentioned index:

using System;
using SCG = System.Collections.Generic;
using System.Linq;
using System.Text;
using C5;

namespace PerformanceMap
{
  class Program
  {
    static void Main(string[] args)
    {
      SCG.Dictionary<int, double> perfMap = new SCG.Dictionary<int, double>();
      int N = 1000;
      //Change the container here and you should be done.
      //C5.HashedArrayList<int> nums = new C5.HashedArrayList<int>();
      C5.HashedLinkedList<int> nums = new C5.HashedLinkedList<int>();
      Enumerable.Range(1, 100).Reverse().ToList().ForEach(n => nums.Add(n));

      for (; N <= 20000; N += 1000)
      {
        DateTime start = DateTime.Now;
        for (int i = 0; i < N; i++)
        {
          nums.Remove(100); 
          nums.Insert(new Random().Next(0,99),100);
        }

        DateTime end = DateTime.Now;
        perfMap.Add(N, end.Subtract(start).TotalMilliseconds );
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]);
      Console.ReadKey();
    }
  }
}

C5.HashedArrayList<T> and C5.HashedLinkedList<T> don't allow duplicate entries. So before inserting 100, we have to remove it. But since these are hash-based collections, removing items from these is a constant time operation. Thus, removing them doesn't have much impact on the overall performance.

The previous code will work for all except LinkedList<T> in the System.Collections.Generic namespace, as it doesn't offer any insert() method to directly insert elements. For LinkedList<T>, we have to use a different code for benchmarking:

using System;
using SCG = System.Collections.Generic;
using System.Linq;
using System.Text;
using C5;

namespace PerformanceMap
{
  class Program
  {
    static void Main(string[] args) 

    {
      SCG.Dictionary<int, double> perfMap = new SCG.Dictionary<int, double>();
      int N = 1000;

      SCG.LinkedList<int> nums = new SCG.LinkedList<int>();
      Enumerable.Range(1, 100).Reverse().ToList().ForEach(n => nums.AddLast(n));

      for (; N <= 20000; N += 1000)
      {
        DateTime start = DateTime.Now;
        for (int i = 0; i < N; i++)
        {
          SCG.LinkedListNode<int> node = nums.Find(new Random().Next(0, 101));
          if (node == null)
          {
            nums.AddFirst(100);
          }
          else
          {
            nums.AddBefore(node, 100);
          }
        }

        DateTime end = DateTime.Now;
        perfMap.Add(N, end.Subtract(start).TotalMilliseconds);
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]);

      Console.ReadKey();
    }
  }
}

As you run these programs for several containers and collect the data and put them on an Excel sheet, you will get an output similar to the following screenshot:

After plotting the data, I got the following graph:

And the following shows how Wordle looks for the previous benchmarking test:

So, if your code demands a lot of insertions at random locations in the container, consider using C5.HashedLinkedList<T>, as that's the fastest you can get.

We can also tell from this tag cloud that List<T> offers one of the worst performances for random insertion. This finding is rather intuitive because containers that implement IList<T> generally use array as the internal data structure to offer native zero-based integer indexing. Therefore, inserting in a IList<T> implementation is actually inserting at a random location in an array. So, all elements after the location where the insertion happens have to be updated. Thus, inserting at a random index in an IList<T> implantation that uses array as the internal data structure is a O(n) operation in the worst case.

How much time does it take to delete a single element in a random location in a list-based collection?

Being able to delete at random locations is also a very important functionality sought by developers. Although it doesn't make much sense in IDictionary implementations and is advisable to avoid; it is almost taken for granted for an IList<T> implantation such as List<T>.

In this experiment, we shall try random deletions from List<T>, LinkedList<T>, C5.LinkedList<T>, C5.ArrayList<T>, C5.HashedLinkedList<T>, and C5.HashedArrayList<T> and see which one wins.

The following is the benchmarking code that will work for LinkedList<T> in System.Collections.Generic:

using System;
using SCG = System.Collections.Generic;
using System.Linq;
using System.Text;
using C5;

namespace PerformanceMap
{
  class Program
  {
    static void Main(string[] args)
    {
      SCG.Dictionary<int, double> perfMap
        = new SCG.Dictionary<int, double>();
      int N = 1000;
      SCG.LinkedList<int> nums = new SCG.LinkedList<int>();

      Enumerable.Range(1, 100).Reverse().ToList()
      .ForEach(n => nums.AddLast(n));


      for (; N <= 20000; N += 1000)
      {
        DateTime start = DateTime.Now;
        int x;
        for (int i = 0; i < N; i++)
        {
          nums.Remove(new Random().Next(0, 100));
        }
        DateTime end = DateTime.Now;
        perfMap.Add(N, end.Subtract(start)
        .TotalMilliseconds);
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]);
      Console.ReadLine();
    }
  }
}

For all other types, just change the highlighted part of the following line:

Enumerable.Range(1, 100).Reverse().ToList()
  .ForEach(n => nums.AddLast(n));

to

Enumerable.Range(1, 100).Reverse().ToList()
  .ForEach(n => nums.Add(n));

After running all the code snippets and collecting the data in the Excel sheet, you shall get a data trend similar to the following:

Using this data, I created the following plot:

I think this is the fanciest looking graph so far! This means that all of these containers offer similar performance when it comes to random deletion. This statement is quite supported in the Wordle art too.

However, array-based collections such as C5.ArrayList<T> and List<T> in System.Collections.Generic are trailing in this functionality too because of the same reason that in case of deletion, in an array worst case performance in O(n) there are n elements in the array.

How fast can we look up an element (or the unique key) in an associative collection?

IDictionary implementations are around for one good reason. They offer very good performance for unique key lookup. In this experiment, we shall see which of the IDictionary implementations is the best when it comes to key lookup. In this experiment, we shall limit our test with the following dictionaries:

Dictionary<TKey,TValue
SortedDictionary<TKey,TValue>
C5.HashDictionary<TKey,TValue> 
C5.TreeDictionary<TKey,TValue>

The following benchmarking code is to be used for C5.HashDictionary<TKey,TValue> and C5.TreeDictionary<TKey,TValue>:

using System;
using SCG = System.Collections.Generic;
using System.Linq;
using System.Text;
using C5;

namespace PerformanceMap
{
  class Program
  {

    static void Main(string[] args)
    {
      string alphabets = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
      SCG.Dictionary<int, double> perfMap =
      new SCG.Dictionary<int, double>();
      int N = 100000;

      C5.HashDictionary<string, int> dic
        = new C5.HashDictionary<string, int>();
      alphabets.ToList()
      .ForEach(a => dic.Add(a.ToString(),
      alphabets.IndexOf(a)));

      for (; N <= 2000000; N += 100000)
      {
        DateTime start = DateTime.Now;
        for (int k = 0; k < N; k++)
        {
          dic.Contains("Z");
        }
        DateTime end = DateTime.Now;
        perfMap.Add(N,
        end.Subtract(start).TotalMilliseconds);
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]); 
      Console.ReadLine();
    }
  }
}

For Dictionary<TKey,TValue> and SortedDictionary<TKey,TValue> in System.Collections.Generic change the following line:

dic.Contains("Z");

to

dic.ContainsKey("Z");

since the method to look up an item in these two IDictionary implementations is ContainsKey().

After executing all the code snippets and entering the data in our Performance Excel Sheet, I got the following output:

Plotting the previous data, I got the following graph:

From the graph, it is evident that C5.TreeDictionary<TKey,TValue> is prohibitively slow and we have to limit our choice between Dictionary<TKey,TValue> and SortedDictionary<TKey,TValue>.

The following screenshot shows how Wordle looks:

There is something about the previous random word. As you can see, Dictionary not only has the highest font size, but it is also the first in the list. The second is SortedDictionary and the last is C5.HashDictionary. C5.TreeDictionary is so slow on key lookup that it almost didn't show up at all. So, the lesson learnt is if you just need a way to associate one type with another and no sorting, use Dictionary else use SortedDictionary.

How fast can we find the union of two or more sets?

Being able to find unions of large sets in a short time is what makes a 21st century algorithm click. We are living in a very interesting time and there is lot of data around us. Our applications have to deal with loads of data every day. But a seemingly simple task of keeping duplicate items at bay is a stupendously difficult task in practice.

To avoid this, we need quick set implementations. The .NET Framework and C5 offer a couple of set implementations. We shall put these containers under a lot of stress and find out how they perform.

The benchmarking code for HashSet<T> and SortedSet<T> available in System.Collections.Generic is as follows:

using System;
using SCG = System.Collections.Generic; 

using System.Linq;
using System.Text;
using C5;

namespace PerformanceMap
{
  class Program
  {
    static void Main(string[] args)
    {

      SCG.HashSet<string> set1 = new SCG.HashSet<string>() 
      { "A", "B", "C", "D", "E" };
      SCG.HashSet<string> set2 = new SCG.HashSet<string>() 
      { "A", "D", "F", "C", "G" };
      SCG.HashSet<string> copyOfSet1 = new SCG.HashSet<string>() 
      { "A", "B", "C", "D", "E" };
      SCG.Dictionary<int, double> perfMap = new SCG.Dictionary<int, double>();
      int N = 100000;

      for (; N <= 2000000; N += 100000)
      {
        DateTime start = DateTime.Now;
        for (int k = 0; k < N; k++)
        {
          set1.UnionWith(set2);
          set1 = copyOfSet1;
        }
        DateTime end = DateTime.Now;
        perfMap.Add(N,
        end.Subtract(start).TotalMilliseconds);
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]);
      Console.ReadLine();
    }
  }
}

The following is the benchmarking code for C5 set implementations. You can replace C5.TreeSet<T> with C5.HashSet<T> in the following code:

using System;
using SCG = System.Collections.Generic;
using System.Linq;
using System.Text;
using C5;

namespace PerformanceMap
{
  class Program
  {
    static void Main(string[] args)
    {
      C5.TreeSet<string> set1 = new C5.TreeSet<string>() 
      { "A", "B", "C", "D", "E" };
      C5.TreeSet<string> set2 = new C5.TreeSet<string>() 
      { "A", "D", "F", "C", "G" };
      C5.TreeSet<string> copyOfSet1 = new C5.TreeSet<string>() 
      { "A", "B", "C", "D", "E" };
      SCG.Dictionary<int, double> perfMap = new SCG.Dictionary<int, double>();
      int N = 100000;

      for (; N <= 2000000; N += 100000) 

      {
        DateTime start = DateTime.Now;
        for (int k = 0; k < N; k++)
        {
          set1.AddAll(set2);
          set1 = copyOfSet1;
        }
        DateTime end = DateTime.Now;
        perfMap.Add(N,
        end.Subtract(start).TotalMilliseconds);
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]);
      Console.ReadLine();
    }
  }
}

The following table has all the data collected from the previous benchmarking:

From the previous data, I got the following plot:

The result of this experiment is supportive of the result of experiment 1. In order to find the union of two sets, it is required to find whether a given element is present in either one or not. So, the container that offers the highest lookup speed for checking whether an element is present or not would intuitively offer the best performance for union also. That's exactly what the result of this experiment shows. HashSet<T> offers the highest lookup speed and also the best union find time.

My Wordle experiment with the data is as follows:

How fast can we find the intersection of two or more sets?

The result of this experiment is exactly the same as that of experiment 7 because intuitively finding a union is similar to finding an intersection. So, in this case, the order of performance is as follows:

HashSet<T> offers the best performance followed by SortedSet<T>, C5.HashSet<T>, and C5.TreeSet.<T>.

The following benchmarking code can be used with C5.HashSet<T> and C5.TreeSet<T>:

using System;
using SCG = System.Collections.Generic; 

using System.Linq;
using System.Text;
using C5;

namespace PerformanceMap
{
  class Program
  {
    static void Main(string[] args)
    {
      C5.TreeSet<string> set1 = new C5.TreeSet<string>() { "A", "B", "C", "D", "E" };
      C5.TreeSet<string> set2 = new C5.TreeSet<string>() 
      { "A", "D", "F", "C", "G" };
      C5.TreeSet<string> copyOfSet1 = new C5.TreeSet<string>() 
      { "A", "B", "C", "D", "E" };
      SCG.Dictionary<int, double> perfMap = new SCG.Dictionary<int, double>();
      int N = 100000;

      for (; N <= 2000000; N += 100000)
      {
        DateTime start = DateTime.Now;
        for (int k = 0; k < N; k++)
        {
          set1.RetainAll(set2);
          //to make sure that we start
          //with the new set implementation 
          //every time
          set1 = copyOfSet1;
        }
        DateTime end = DateTime.Now;
        perfMap.Add(N,
        end.Subtract(start).TotalMilliseconds);
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]);
      Console.ReadLine();
    }
  }
}

For HashSet<T> and SortedSet<T> in System.Collections.Generic just change the following line:

set1.RetainAll(set2);

to

set1.IntersectWith(set2);

As the results show a trend similar to the previous experiment, I didn't include the table values and the Wordle for this experiment.

How fast can we check whether a set is a subset of another set or not?

We shall benchmark the same four set implementations here.

The following code is to be used for HashSet<T> and SortedSet<T>:

using System;
using SCG = System.Collections.Generic;
using System.Linq;
using System.Text;
using C5;

namespace PerformanceMap
{
  class Program
  {
    static void Main(string[] args)
    {
      SCG.HashSet<string> set1 = new SCG.HashSet<string>() 
      { "A", "B", "C", "D", "E" };
      SCG.HashSet<string> set2 = new SCG.HashSet<string>() 
      { "A", "D", "F", "C", "G" };
      SCG.HashSet<string> copyOfSet1 = new SCG.HashSet<string>() 
      { "A", "B", "C", "D", "E" };
      SCG.Dictionary<int, double> perfMap = new SCG.Dictionary<int, double>();
      int N = 100000;

      for (; N <= 2000000; N += 100000)
      {
        DateTime start = DateTime.Now;
        for (int k = 0; k < N; k++)
        set1.IsSubsetOf(set2);
        DateTime end = DateTime.Now;
        perfMap.Add(N, end.Subtract(start).TotalMilliseconds);
      }
      foreach (int k in perfMap.Keys)
      Console.WriteLine(k + " " + perfMap[k]);

      Console.ReadLine();
    }
  }
}

For C5.HashSet<T> and C5.TreeSet<T> the benchmarking code has to be modified by changing the following line:

set1.IsSubsetOf(set2); 

to

set1.ContainsAll(set2);

When the details are copied to our performance spreadsheet, we get data similar to the following screenshot:

Plotting the previous data, I got the following graph:

And the Wordle for this benchmarking looks similar to the following screenshot:

We have come to an end with our experiments for this chapter. I hope you got an idea how to benchmark your collections for several needs. It is very interesting that while C5.TreeSet<T> is almost unusable for a union or an intersection, it outperforms other set implementations when it comes to the finding subset functionality.

So, the lesson learnt is one size doesn't fit all. Learn what works best for the job at hand by benchmarking it against several options.

We have come a long way and now know with confidence which container is better in a given situation than another, even though theoretical time complexity for the mentioned operation in question is the same.

I hope the guidelines mentioned in this chapter will help you steer clear of pitfalls that can appear easily while dealing with code involving Generics.

You can download an Excel Workbook with all these benchmarking details from the book's website.