foreach with Arrays

Listing 15.3 demonstrates a simple foreach loop iterating over an array of integers and then printing out each integer to the console.

Listing 15.3: foreach with Arrays

Click here to view code image

int[] array = new int[]{1, 2, 3, 4, 5, 6};

foreach (int item in array)
{
    Console.WriteLine(item);
}

From this code, the C# compiler creates a CIL equivalent of the for loop, as shown in Listing 15.4.

Listing 15.4: Compiled Implementation of foreach with Arrays

Click here to view code image

int[] tempArray;
int[] array = new int[]{1, 2, 3, 4, 5, 6};

tempArray = array;
for (int counter = 0; (counter < tempArray.Length); counter++)
{
  int item = tempArray[counter];

  Console.WriteLine(item);
}

In this example, note that foreach relies on support for the Length property and the index operator ([]). With the Length property, the C# compiler can use the for statement to iterate through each element in the array.

foreach with IEnumerable<T>

Although the code shown in Listing 15.4 works well on arrays where the length is fixed and the index operator is always supported, not all types of collections have a known number of elements. Furthermore, many of the collection classes, including the Stack<T>, Queue<T>, and Dictionary<TKey, TValue> classes, do not support retrieving elements by index. Therefore, a more general approach of iterating over collections of elements is needed. The iterator pattern provides this capability. Assuming you can determine the first and next elements, knowing the count and supporting retrieval of elements by index are unnecessary.

The System.Collections.Generic.IEnumerator<T> and nongeneric System.Collections.IEnumerator interfaces are designed to enable the iterator pattern for iterating over collections of elements, rather than the length–index pattern shown in Listing 15.4. A class diagram of their relationships appears in Figure 15.1.

IEnumerator, which IEnumerator<T> derives from, includes three members. The first is bool MoveNext(). Using this method, you can move from one element within the collection to the next, while at the same time detecting when you have enumerated through every item. The second member, a read-only property called Current, returns the element currently in process. Current is overloaded in IEnumerator<T>, providing a type-specific implementation of it. With these two members of the collection class, it is possible to iterate over the collection by simply using a while loop, as demonstrated in Listing 15.5. (The Reset() method usually throws a NotImplementedException, so it should never be called. If you need to restart an enumeration, just create a fresh enumerator.)

Listing 15.5: Iterating over a Collection Using while

Click here to view code image

System.Collections.Generic.Stack<int> stack =
  new System.Collections.Generic.Stack<int>();
int number;
// ...

// This code is conceptual, not the actual code
while (stack.MoveNext())
{
  number = stack.Current;
  Console.WriteLine(number);
}

In Listing 15.5, the MoveNext() method returns false when it moves past the end of the collection. This replaces the need to count elements while looping.

Listing 15.5 uses a System.Collections.Generic.Stack<T> as the collection type. Numerous other collection types exist; this is just one example. The key trait of Stack<T> is its design as a last in, first out (LIFO) collection. Notice that the type parameter T identifies the type of all items within the collection. Collecting one type of object within a collection is a key characteristic of a generic collection. The programmer must know the data type within the collection when adding, removing, or accessing items within the collection.

The preceding example shows the gist of the C# compiler output, but it doesn’t actually compile that way because it omits two important details concerning the implementation: interleaving and error handling.

System.Collections.Generic.Stack<int> stack =
  new System.Collections.Generic.Stack<int>();
int number;
System.Collections.Generic.Stack<int>.Enumerator
  enumerator;

// ...

// If IEnumerable<T> is implemented explicitly,
// then a cast is required:
// ((IEnumerable<int>)stack).GetEnumerator();
enumerator = stack.GetEnumerator();
while (enumerator.MoveNext())
{
  number = enumerator.Current;
  Console.WriteLine(number);
}

Do Not Modify Collections during foreach Iteration

Chapter 4 showed that the compiler prevents assignment of the foreach variable (number). As is demonstrated in Listing 15.7, an assignment to number would not change the collection element itself, so the C# compiler prevents such an assignment altogether.

In addition, neither the element count within a collection nor the items themselves can generally be modified during the execution of a foreach loop. If, for example, you called stack.Push(42) inside the foreach loop, it would be ambiguous whether the iterator should ignore or incorporate the change to stack—in other words, whether iterator should iterate over the newly added item or ignore it and assume the same state as when it was instantiated.

Because of this ambiguity, an exception of type System.InvalidOperationException is generally thrown upon accessing the enumerator if the collection is modified within a foreach loop. This exception reports that the collection was modified after the enumerator was instantiated.

Filtering with Where()

To filter out data from a collection, we need to provide a filter method that returns true or false, indicating whether or not a particular element should be included. A delegate expression that takes an argument and returns a Boolean value is called a predicate, and a collection’s Where() method depends on predicates for identifying filter criteria, as shown in Listing 15.10. (Technically, the result of the Where() method is an object that encapsulates the operation of filtering a given sequence with a given predicate.) The results appear in Output 15.2.

Listing 15.10: Filtering with System.Linq.Enumerable.Where()

Click here to view code image

using System;
using System.Collections.Generic;
using System.Linq;

class Program
{
  static void Main()
  {
      IEnumerable<Patent> patents = PatentData.Patents;
      patents = patents.Where(                                 
          patent => patent.YearOfPublication.StartsWith("18"));
      Print(patents);
  }

  // ...
}

Output 15.2

Phonograph (1877)
Kinetoscope (1888)
Electrical Telegraph (1837)
Steam Locomotive (1815)

Notice that the code assigns the output of the Where() call back to IEnumerable<T>. In other words, the output of IEnumerable<T>.Where() is a new IEnumerable<T> collection. In Listing 15.10, it is IEnumerable<Patent>.

Less obvious is that the Where() expression argument has not necessarily been executed at assignment time. This is true for many of the standard query operators. In the case of Where(), for example, the expression is passed into the collection and “saved” but not executed. Instead, execution of the expression occurs only when it is necessary to begin iterating over the items within the collection. For example, a foreach loop, such as the one in Print() (in Listing 15.9), can trigger the expression to be evaluated for each item within the collection. At least conceptually, the Where() method should be understood as a means of specifying the query regarding what appears in the collection, not the actual work involved with iterating over the items to produce a new collection with potentially fewer items.

Projecting with Select()

Since the output from the IEnumerable<T>.Where() method is a new IEnumerable<T> collection, it is possible to again call a standard query operator on the same collection. For example, rather than just filtering the data from the original collection, we could transform the data (see Listing 15.11).

Listing 15.11: Projection with System.Linq.Enumerable.Select()

Click here to view code image

using System;
using System.Collections.Generic;
using System.Linq;

class Program
{
  static void Main()
  {
      IEnumerable<Patent> patents = PatentData.Patents;
      IEnumerable<Patent> patentsOf1800 = patents.Where(
          patent => patent.YearOfPublication.StartsWith("18"));
      IEnumerable<string> items = patentsOf1800.Select(                        
          patent => patent.ToString());                                        

      Print(items);
  }

  // ...
}

In Listing 15.11, we create a new IEnumerable<string> collection. In this case, it just so happens that adding the Select() call doesn’t change the output—but only because Print()’s Console.WriteLine() call used ToString() anyway. Obviously, a transform still occurred on each item from the Patent type of the original collection to the string type of the items collection.

Consider the example using System.IO.FileInfo in Listing 15.12.

Listing 15.12: Projection with System.Linq.Enumerable.Select() and new

Click here to view code image

// ...
IEnumerable<string> fileList = Directory.GetFiles(
    rootDirectory, searchPattern);
IEnumerable<FileInfo> files = fileList.Select(
    file => new FileInfo(file));
// ...

Here fileList is of type IEnumerable<string>. However, using the projection offered by Select, we can transform each item in the collection to a System.IO.FileInfo object.

Lastly, capitalizing on tuples, we can create an IEnumerable<T> collection where T is a tuple (see Listing 15.13 and Output 15.3).

Listing 15.13: Projection to Tuple

Click here to view code image

// ...
IEnumerable<string> fileList = Directory.EnumerateFiles(
    rootDirectory, searchPattern);
IEnumerable<(string FileName, long Size)> items = fileList.Select(                        
    file =>                                                                               
    {                                                                                     
        FileInfo fileInfo = new FileInfo(file);                                           
        return (                                                                          
            FileName: fileInfo.Name,                                                      
            Size: fileInfo.Length                                                         
        );                                                                                
    });                                                                                   
// ...

Output 15.3

Click here to view code image

FileName = AssemblyInfo.cs, Size = 1704
FileName = CodeAnalysisRules.xml, Size = 735
FileName = CustomDictionary.xml, Size = 199
FileName = EssentialCSharp.sln, Size = 40415
FileName = EssentialCSharp.suo, Size = 454656
FileName = EssentialCSharp.vsmdi, Size = 499
FileName = EssentialCSharp.vssscc, Size = 256
FileName = intelliTechture.ConsoleTester.dll, Size = 24576
FileName = intelliTechture.ConsoleTester.pdb, Size = 30208

The output of an anonymous type automatically shows the property names and their values as part of the generated ToString() method associated with the anonymous type.

Projection using the Select() method is very powerful. We already saw how to filter a collection vertically (reducing the number of items in the collection) using the Where() standard query operator. Now, via the Select() standard query operator, we can also reduce the collection horizontally (making fewer columns) or transform the data entirely. In combination, Where() and Select() provide a means for extracting only those pieces of the original collection that are desirable for the current algorithm. These two methods alone provide a powerful collection manipulation API that would otherwise result in significantly more—and less readable—code.

Counting Elements with Count()

Another query frequently performed on a collection of items is to retrieve the count. To support this type of query, LINQ includes the Count() extension method.

Listing 15.15 demonstrates that Count() is overloaded to simply count all elements (no parameters) or to take a predicate that counts only items identified by the predicate expression.

Listing 15.15: Counting Items with Count()

Click here to view code image

using System;
using System.Collections.Generic;
using System.Linq;

class Program
{
  static void Main()
  {
      IEnumerable<Patent> patents = PatentData.Patents;
      Console.WriteLine($"Patent Count: { patents.Count() }");
      Console.WriteLine($@"Patent Count in 1800s: {
          patents.Count(patent =>
              patent.YearOfPublication.StartsWith("18"))
      }");
  }

  // ...
}

In spite of the apparent simplicity of the Count() statement, IEnumerable<T> has not changed, so the executed code still iterates over all the items in the collection. Whenever a Count property is directly available on the collection, it is preferable to use that rather than LINQ’s Count() method (a subtle difference). Fortunately, ICollection<T> includes the Count property, so code that calls the Count() method on a collection that supports ICollection<T> will cast the collection and call Count directly. However, if ICollection<T> is not supported, Enumerable.Count() will proceed to enumerate all the items in the collection rather than call the built-in Count mechanism. If the purpose of checking the count is just to see whether it is greater than zero (if(patents.Count() > 0){...}), the preferred approach would be to use the Any() operator (if(patents.Any()){...}). Any() attempts to iterate over only one of the items in the collection to return a true result, rather than iterating over the entire sequence.

Deferred Execution

One of the most important concepts to remember when using LINQ is deferred execution. Consider the code in Listing 15.16 and the corresponding output in Output 15.4.

Listing 15.16: Filtering with System.Linq.Enumerable.Where()

Click here to view code image

using System;
using System.Collections.Generic;
using System.Linq;

// ...

      IEnumerable<Patent> patents = PatentData.Patents;
      bool result;
      patents = patents.Where(
          patent =>
          {
              if (result =
                  patent.YearOfPublication.StartsWith("18"))
              {
                  // Side effects like this in a predicate
                  // are used here to demonstrate a
                  // principle and should generally be
                  // avoided
                  Console.WriteLine("	" + patent);
              }
              return result;
          });
      Console.WriteLine("1. Patents prior to the 1900s are:");
      foreach (Patent patent in patents)
      {
      }

      Console.WriteLine();
      Console.WriteLine(
          "2. A second listing of patents prior to the 1900s:");
      Console.WriteLine(
          $@"   There are { patents.Count()
              } patents prior to 1900.");

      Console.WriteLine();
      Console.WriteLine(
          "3. A third listing of patents prior to the 1900s:");
      patents = patents.ToArray();
      Console.Write("   There are ");
      Console.WriteLine(
          $"{ patents.Count() } patents prior to 1900.");

// ...

Output 15.4

Click here to view code image

1. Patents prior to the 1900s are:
        Phonograph (1877)
        Kinetoscope (1888)
        Electrical Telegraph (1837)
        Steam Locomotive (1815)

2. A second listing of patents prior to the 1900s:
        Phonograph (1877)
        Kinetoscope (1888)
        Electrical Telegraph (1837)
        Steam Locomotive (1815)
   There are 4 patents prior to 1900.

3. A third listing of patents prior to the 1900s:
        Phonograph (1877)
        Kinetoscope (1888)
        Electrical Telegraph (1837)
        Steam Locomotive (1815)
   There are 4 patents prior to 1900.

Notice that Console.WriteLine("1. Patents prior...) executes before the lambda expression. This characteristic is very important to recognize because it is not obvious to those who are unaware of its importance. In general, predicates should do exactly one thing—evaluate a condition—and should not have any side effects (even printing to the console, as in this example).

To understand what is happening, recall that lambda expressions are delegates—references to methods—that can be passed around. In the context of LINQ and standard query operators, each lambda expression forms part of the overall query to be executed.

At the time of declaration, lambda expressions are not executed. In fact, it isn’t until the lambda expressions are invoked that the code within them begins to execute. Figure 15.2 shows the sequence of operations.

As Figure 15.2 shows, three calls in Listing 15.14 trigger the lambda expression, and each time it is fairly implicit. If the lambda expression were expensive (such as a call to a database), it would therefore be important to minimize the lambda expression’s execution.

First, the execution is triggered within the foreach loop. As described earlier in the chapter, the foreach loop breaks down into a MoveNext() call, and each call results in the lambda expression’s execution for each item in the original collection. While iterating, the runtime invokes the lambda expression for each item to determine whether the item satisfies the predicate.

Second, a call to Enumerable’s Count() (the function) triggers the lambda expression for each item once more. Again, this is subtle behavior because Count (the property) is very common on collections that have not been queried with a standard query operator.

Third, the call to ToArray() (or ToList(), ToDictionary(), or ToLookup()) evaluates the lambda expression for each item. However, converting the collection with one of these “To” methods is extremely helpful. Doing so returns a collection on which the standard query operator has already executed. In Listing 15.14, the conversion to an array means that when Length is called in the final Console.WriteLine(), the underlying object pointed to by patents is, in fact, an array (which obviously implements IEnumerable<T>); in turn, System.Array’s implementation of Length is called and not System.Linq.Enumerable’s implementation. Consequently, following a conversion to one of the collection types returned by a “To” method, it is generally safe to work with the collection (until another standard query operator is called). However, be aware that this will bring the entire result set into memory (it may have been backed by a database or file prior to this step). Furthermore, the “To” method will take a snapshot of the underlying data, so that no fresh results will be returned upon requerying the “To” method result.

We strongly encourage you to review the sequence diagram in Figure 15.2 along with the corresponding code and recognize that the deferred execution of standard query operators can result in extremely subtle triggering of the standard query operators; therefore, developers should use caution and seek to avoid unexpected calls. The query object represents the query, not the results. When you ask the query for the results, the whole query executes (perhaps even again) because the query object doesn’t know that the results will be the same as they were during a previous execution (if one existed).

Sorting with OrderBy() and ThenBy()

Another common operation on a collection is to sort it. Sorting involves a call to System.Linq.Enumerable’s OrderBy(), as shown in Listing 15.17 and Output 15.5.

Listing 15.17: Ordering with System.Linq.Enumerable.OrderBy()/ThenBy()

Click here to view code image

using System;
using System.Collections.Generic;
using System.Linq;

// ...

      IEnumerable<Patent> items;
      Patent[] patents = PatentData.Patents;
      items = patents.OrderBy(
          patent => patent.YearOfPublication).ThenBy(
          patent => patent.Title);
      Print(items);
      Console.WriteLine();
      items = patents.OrderByDescending(
          patent => patent.YearOfPublication).ThenByDescending(
          patent => patent.Title);
      Print(items);

// ...

Output 15.5

Click here to view code image

Bifocals (1784)
Steam Locomotive (1815)
Electrical Telegraph (1837)
Phonograph (1877)
Kinetoscope (1888)
Flying Machine (1903)
Backless Brassiere (1914)

Droplet Deposition Apparatus (1989)
Droplet Deposition Apparatus (1989)
Backless Brassiere (1914)
Flying Machine (1903)
Kinetoscope (1888)
Phonograph (1877)
Electrical Telegraph (1837)
Steam Locomotive (1815)
Bifocals (1784)

The OrderBy() call takes a lambda expression that identifies the key on which to sort. In Listing 15.17, the initial sort uses the year in which the patent was published.

Notice that the OrderBy() call takes only a single parameter, keySelector, to sort on. To sort on a second column, it is necessary to use a different method: ThenBy(). Similarly, code would use ThenBy() for any additional sorting.

OrderBy() returns an IOrderedEnumerable<T> interface, not an IEnumerable<T>. Furthermore, IOrderedEnumerable<T> derives from IEnumerable<T>, so all the standard query operators (including OrderBy()) are available on the OrderBy() return. However, repeated calls to OrderBy() would undo the work of the previous call such that the end result would sort by only the keySelector in the final OrderBy() call. For this reason, you should be careful not to call OrderBy() on a previous OrderBy() call.

Instead, you should specify additional sorting criteria using ThenBy(). Although ThenBy() is an extension method, it is not an extension of IEnumerable<T> but rather of IOrderedEnumerable<T>. The method, also defined on System.Linq.Extensions.Enumerable, is declared as follows:

Click here to view code image

public static IOrderedEnumerable<TSource>
  ThenBy<TSource, TKey>(
    this IOrderedEnumerable<TSource> source,
    Func<TSource, TKey> keySelector)

In summary, use OrderBy() first, followed by zero or more calls to ThenBy() to provide additional sorting “columns.” The methods OrderByDescending() and ThenByDescending() provide the same functionality except that they sort items in descending order. Mixing and matching ascending and descending methods is not a problem, but if sorting items further, you would use a ThenBy() call (either ascending or descending).

Two more important notes about sorting are warranted. First, the actual sort doesn’t occur until you begin to access the members in the collection, at which point the entire query is processed. You can’t sort unless you have all the items to sort, because you can’t determine whether you have the first item. The fact that sorting is delayed until you begin to access the members is due to deferred execution, as described earlier in this chapter. Second, each subsequent call to sort the data (e.g., Orderby() followed by ThenBy() followed by ThenByDescending()) does involve additional calls to the keySelector lambda expression of the earlier sorting calls. In other words, a call to OrderBy() will call its corresponding keySelector lambda expression once you iterate over the collection. Furthermore, a subsequent call to ThenBy() will again make calls to OrderBy()’s keySelector.

Output 15.6

Click here to view code image

Corporate
Human Resources
Engineering
Information Technology
Philanthropy
Marketing
Mark Michaelis (Chief Computer Nerd)
Michael Stokesbary (Senior Computer Wizard)
Brian Jones (Enterprise Integration Guru)
Anne Beard (HR Director)
Pat Dever (Enterprise Architect)
Kevin Bost (Programmer Extraordinaire)
Thomas Heavey (Software Architect)
Eric Edmonds (Philanthropy Coordinator)

We use this data in the example in the following section on joining data.

Performing an Inner Join with Join()

In the world of objects on the client side, relationships between objects are generally already set up. For example, the relationship between files and the directories in which they reside are preestablished with the DirectoryInfo.GetFiles() method and the FileInfo.Directory method, respectively. Frequently, however, this is not the case with data being loaded from nonobject stores. Instead, the data needs to be joined together so that you can navigate from one type of object to the next in a way that makes sense for the data.

Consider the example of employees and company departments. In Listing 15.19, we join each employee to his or her department and then list each employee with his or her corresponding department. Since each employee belongs to only one (and exactly one) department, the total number of items in the list is equal to the total number of employees—each employee appears only once (each employee is said to be normalized). Output 15.7 shows the results.

Listing 15.19: An Inner Join Using System.Linq.Enumerable.Join()

Click here to view code image

using System;
using System.Linq;

// ...

      Department[] departments = CorporateData.Departments;
      Employee[] employees = CorporateData.Employees;

      IEnumerable<(int Id, string Name, string Title,
              Department Department)> items =
          employees.Join(
              departments,
              employee => employee.DepartmentId,
              department => department.Id,
              (employee, department) => (
                  employee.Id,
                  employee.Name,
                  employee.Title,
                  department
              ));

      foreach (var item in items)
      {
          Console.WriteLinae(
              $"{ item.Name } ({ item.Title })");
          Console.WriteLine("	" + item.Department);
      }

// ...

Output 15.7

Click here to view code image

Mark Michaelis (Chief Computer Nerd)
        Corporate
Michael Stokesbary (Senior Computer Wizard)
        Engineering
Brian Jones (Enterprise Integration Guru)
        Engineering
Anne Beard (HR Director)
        Human Resources
Pat Dever (Enterprise Architect)
        Information Technology
Kevin Bost (Programmer Extraordinaire)
        Engineering
Thomas Heavey (Software Architect)
        Engineering
Eric Edmonds (Philanthropy Coordinator)
        Philanthropy

The first parameter for Join() has the name inner. It specifies the collection, departments, that employees joins to. The next two parameters are lambda expressions that specify how the two collections will connect. employee => employee.DepartmentId (with a parameter name of outerKeySelector) identifies that on each employee, the key will be DepartmentId. The next lambda expression (department =>department.Id) specifies the Department’s Id property as the key—in other words, for each employee, join a department where employee.DepartmentId equals department.Id. The last parameter is the resultant item that is selected. In this case, it is a tuple with Employee’s Id, Name, and Title, as well as a Department property with the joined department object.

Notice in the output that Engineering appears multiple times—once for each employee in CorporateData. In this case, the Join() call produces a Cartesian product between all the departments and all the employees, such that a new record is created for every case where a record exists in both collections and the specified department IDs are the same. This type of join is an inner join.

The data could also be joined in reverse, such that department joins to each employee to list each department-to-employee match. Notice that the output includes more records than there are departments: There are multiple employees for each department, and the output is a record for each match. As we saw before, the Engineering department appears multiple times, once for each employee.

The code in Listing 15.20 (which produces Output 15.8) is similar to that in Listing 15.19, except that the objects, Departments and Employees, are reversed. The first parameter to Join() is employees, indicating what departments joins to. The next two parameters are lambda expressions that specify how the two collections will connect: department => department.Id for departments and employee => employee.DepartmentId for employees. As before, a join occurs whenever department.Id equals employee.EmployeeId. The final tuple parameter specifies a class with int Id, string Name, and Employee Employee items. (Specifying the names in the expression is optional but used here for clarity.)

Listing 15.20: Another Inner Join with System.Linq.Enumerable.Join()

Click here to view code image

using System;
using System.Linq;

// ...

      Department[] departments = CorporateData.Departments;
      Employee[] employees = CorporateData.Employees;

      IEnumerable<(long Id, string Name, Employee Employee)> items =
          departments.Join(
              employees,
              department => department.Id,
              employee => employee.DepartmentId,
              (department, employee) => (
                  department.Id,
                  department.Name,
                  Employee: employee)
          );

      foreach (var item in items)
      {
          Console.WriteLine(item.Name);
          Console.WriteLine("	" + item.Employee);
      }

// ...

Output 15.8

Click here to view code image

Corporate
        Mark Michaelis (Chief Computer Nerd)
Human Resources
        Anne Beard (HR Director)
Engineering
        Michael Stokesbary (Senior Computer Wizard)
Engineering
        Brian Jones (Enterprise Integration Guru)
Engineering
        Kevin Bost (Programmer Extraordinaire)
Engineering
        Thomas Heavey (Software Architect)
Information Technology
        Pat Dever (Enterprise Architect)
Philanthropy
        Eric Edmonds (Philanthropy Coordinator)

In addition to ordering and joining a collection of objects, you might want to group objects with like characteristics. For the employee data, you might want to group employees by department, region, job title, and so forth. Listing 15.21 shows an example of how to do this with the GroupBy() standard query operator (see Output 15.9 to view the results).

Listing 15.21: Grouping Items Using System.Linq.Enumerable.GroupBy()

Click here to view code image

using System;
using System.Linq;

// ...

      IEnumerable<Employee> employees = CorporateData.Employees;

      IEnumerable<IGrouping<int, Employee>> groupedEmployees =
        employees.GroupBy((employee) => employee.DepartmentId);
      foreach(IGrouping<int, Employee> employeeGroup in
          groupedEmployees)
      {
          Console.WriteLine();
          foreach(Employee employee in employeeGroup)
          {
              Console.WriteLine("	" + employee);
          }
          Console.WriteLine(
            "	Count: " + employeeGroup.Count());
      }
// ...

Output 15.9

Click here to view code image

Mark Michaelis (Chief Computer Nerd)
    Count: 1
Michael Stokesbary (Senior Computer Wizard)
Brian Jones (Enterprise Integration Guru)
Kevin Bost (Programmer Extraordinaire)
Thomas Heavey (Software Architect)
    Count: 4
Anne Beard (HR Director)
    Count: 1
Pat Dever (Enterprise Architect)
    Count: 1
Eric Edmonds (Philanthropy Coordinator)
    Count: 1

Note that the items output from a GroupBy() call are of type IGrouping<TKey, TElement>, which has a property for the key that the query is grouping on (employee.DepartmentId). However, it does not have a property for the items within the group. Rather, IGrouping<TKey, TElement> derives from IEnumerable<T>, allowing for enumeration of the items within the group using a foreach statement or for aggregating the data into something such as a count of items (employeeGroup.Count()).

Implementing a One-to-Many Relationship with GroupJoin()

Listings 15.19 and 15.20 are virtually identical. Either Join() call could have produced the same output just by changing the tuple definition. When trying to create a list of employees, Listing 15.19 provides the correct result. Department ends up as an item of both tuples representing the joined employee. However, Listing 15.20 is not ideal. Given support for collections, a more preferable representation of a department would have a collection of employees rather than a single tuple for each department–employee relationship. Listing 15.22 demonstrates the creation of such a child collection; Output 15.10 shows the preferred output.

Listing 15.22: Creating a Child Collection with System.Linq.Enumerable.GroupJoin()

Click here to view code image

using System;
using System.Linq;

// ...

      Department[] departments = CorporateData.Departments;
      Employee[] employees = CorporateData.Employees;

      IEnumerable<(long Id, string Name, IEnumerable<Employee> Employees)> items =

          departments.GroupJoin(
              employees,
              department => department.Id,
              employee => employee.DepartmentId,
              (department, departmentEmployees) => (
                  department.Id,
                  department.Name,
                  departmentEmployees
              ));

      foreach (
          (_, string name, IEnumerable<Employee> employeeCollection) in items)
      {
          Console.WriteLine(name);
          foreach (Employee employee in employeeCollection)
          {
              Console.WriteLine("	" + employee);
          }
      }

// ...

Output 15.10

Click here to view code image

Corporate
        Mark Michaelis (Chief Computer Nerd)
Human Resources
        Anne Beard (HR Director)
Engineering
        Michael Stokesbary (Senior Computer Wizard)
        Brian Jones (Enterprise Integration Guru)
        Kevin Bost (Programmer Extraordinaire)
        Thomas Heavey (Software Architect)
Information Technology
        Pat Dever (Enterprise Architect)
Philanthropy
        Eric Edmonds (Philanthropy Coordinator)

To achieve the preferred result, we use System.Linq.Enumerable’s GroupJoin() method. The parameters are the same as those in Listing 15.19, except for the final tuple selected. In Listing 15.19, the lambda expression is of type Func<Department, IEnumerable<Employee>, (long Id, string Name, IEnumerable<Employee> Employees)>. Notice that we use the second type argument (IEnumerable<Employee>) to project the collection of employees for each department onto the resultant department tuple; thus each department in the resulting collection includes a list of the employees.

(Readers familiar with SQL will notice that, unlike Join(), GroupJoin() doesn’t have a SQL equivalent because the data returned by SQL is record based, not hierarchical.)

Calling SelectMany()

On occasion, you may have collections of collections. Listing 15.24 provides an example of such a scenario. The teams array contains two teams, each with a string array of players.

Listing 15.24: Calling SelectMany()

Click here to view code image

using System;
using System.Collections.Generic;
using System.Linq;

// ...

      (string Team, string[] Players)[] worldCup2006Finalists = new[]
      {
          (
              TeamName: "France",
              Players: new string[]
              {
                  "Fabien Barthez", "Gregory Coupet",
                  "Mickael Landreau", "Eric Abidal",
                  "Jean-Alain Boumsong", "Pascal Chimbonda",
                  "William Gallas", "Gael Givet",
                  "Willy Sagnol", "Mikael Silvestre",
                  "Lilian Thuram", "Vikash Dhorasoo",
                  "Alou Diarra", "Claude Makelele",
                  "Florent Malouda", "Patrick Vieira",
                  "Zinedine Zidane", "Djibril Cisse",
                  "Thierry Henry", "Franck Ribery",
                  "Louis Saha", "David Trezeguet",
                  "Sylvain Wiltord",
              }
          ),
          (
              TeamName: "Italy",
              Players: new string[]
              {
                  "Gianluigi Buffon", "Angelo Peruzzi",
                  "Marco Amelia", "Cristian Zaccardo",
                  "Alessandro Nesta", "Gianluca Zambrotta",
                  "Fabio Cannavaro", "Marco Materazzi",
                  "Fabio Grosso", "Massimo Oddo",
                  "Andrea Barzagli", "Andrea Pirlo",
                  "Gennaro Gattuso", "Daniele De Rossi",
                  "Mauro Camoranesi", "Simone Perrotta",
                  "Simone Barone", "Luca Toni",
                  "Alessandro Del Piero", "Francesco Totti",
                  "Alberto Gilardino", "Filippo Inzaghi",
                  "Vincenzo Iaquinta",
              }
          )
      };

      IEnumerable<string> players =
          worldCup2006Finalists.SelectMany(
              team => team.Players);

      Print(players);

// ...

The output from this listing has each player’s name displayed on its own line in the order in which it appears in the code. The difference between Select() and SelectMany() is that Select() would return two items, one corresponding to each item in the original collection. Select() may project out a transform from the original type, but the number of items would not change. For example, teams.Select(team => team.Players) will return an IEnumerable<string[]>.

In contrast, SelectMany() iterates across each item identified by the lambda expression (the array selected by Select() earlier) and hoists out each item into a new collection that includes a union of all items within the child collection. Instead of two arrays of players, SelectMany() combines each array selected and produces a single collection of all items.

More Standard Query Operators

Listing 15.25 shows code that uses some of the simpler APIs enabled by Enumerable; Output 15.12 shows the results.

Listing 15.25: More System.Linq.Enumerable Method Calls

Click here to view code image

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

class Program
{
  static void Main()
  {
      IEnumerable<object> stuff =
        new object[] { new object(), 1, 3, 5, 7, 9,
            ""thing"", Guid.NewGuid() };
      Print("Stuff: {0}", stuff);
      IEnumerable<int> even = new int[] { 0, 2, 4, 6, 8 };
      Print("Even integers: {0}", even);

      IEnumerable<int> odd = stuff.OfType<int>();
      Print("Odd integers: {0}", odd);

      IEnumerable<int> numbers = even.Union(odd);
      Print("Union of odd and even: {0}", numbers);

      Print("Union with even: {0}", numbers.Union(even));
      Print("Concat with odd: {0}", numbers.Concat(odd));
      Print("Intersection with even: {0}",
          numbers.Intersect(even));
      Print("Distinct: {0}", numbers.Concat(odd).Distinct());
      if (!numbers.SequenceEqual(
          numbers.Concat(odd).Distinct()))
      {
          throw new Exception("Unexpectedly unequal");
      }
      else
      {
          Console.WriteLine(
              @"Collection ""SequenceEquals""" +
                    " numbers.Concat(odd).Distinct())")
      }
      Print("Reverse: {0}", numbers.Reverse());
      Print("Average: {0}", numbers.Average());
      Print("Sum: {0}", numbers.Sum());
      Print("Max: {0}", numbers.Max());
      Print("Min: {0}", numbers.Min());
  }

  private static void Print<T>(
          string format, IEnumerable<T> items)
          where T: notnull =>
      Console.WriteLine(format, string.Join(
          ", ", items));
}

Output 15.12

Click here to view code image

Stuff: System.Object, 1, 3, 5, 7, 9, "thing"
24c24a41-ee05-41b9-958e-50dd12e3981e
Even integers: 0, 2, 4, 6, 8
Odd integers: 1, 3, 5, 7, 9
Union of odd and even: 0, 2, 4, 6, 8, 1, 3, 5, 7, 9
Union with even: 0, 2, 4, 6, 8, 1, 3, 5, 7, 9
Concat with odd: 0, 2, 4, 6, 8, 1, 3, 5, 7, 9, 1, 3, 5, 7, 9
Intersection with even: 0, 2, 4, 6, 8
Distinct: 0, 2, 4, 6, 8, 1, 3, 5, 7, 9
Collection "SequenceEquals" numbers.Concat(odd).Distinct())
Reverse: 9, 7, 5, 3, 1, 8, 6, 4, 2, 0
Average: 4.5
Sum: 45
Max: 9
Min: 0

None of the API calls in Listing 15.25 requires a lambda expression. Tables 15.1 and 15.2 describe each method and provide an example. Included in System.Linq.Enumerable is a collection of aggregate functions that enumerate the collection and calculate a result (shown in Table 15.2). Count is one example of an aggregate function already shown in the chapter.

Note that each method listed in Tables 15.1 and 15.2 will trigger deferred execution.

Anonymous Types

Anonymous types are data types that are declared by the compiler rather than through the explicit class definitions introduced in Chapter 6. As with anonymous functions, when the compiler sees an anonymous type, it does the work to make that class for you and then lets you use it as though you had declared it explicitly. Listing 15.26 shows such a declaration.

Listing 15.26: Implicit Local Variables with Anonymous Types

Click here to view code image

using System;

class Program
{
  static void Main()
  {
      var patent1 =                                                      
          new                                                            
          {                                                              
              Title = "Bifocals",                                        
              YearOfPublication = "1784"                                 
          };                                                             
      var patent2 =                                                      
          new                                                            
          {                                                              
              Title = "Phonograph",                                      
              YearOfPublication = "1877"                                 
          };                                                             
      var patent3 =                                                      
          new                                                            
          {                                                              
              patent1.Title,                                             
              // Renamed to show property naming.                        
              Year = patent1.YearOfPublication                           
          };                                                             

      Console.WriteLine(
          $"{ patent1.Title } ({ patent1.YearOfPublication })");
      Console.WriteLine(
          $"{ patent2.Title } ({ patent2.YearOfPublication })");

      Console.WriteLine();
      Console.WriteLine(patent1);
      Console.WriteLine(patent2);

      Console.WriteLine();
      Console.WriteLine(patent3);
  }
}

The corresponding output is shown in Output 15.13.

Output 15.13

Click here to view code image

Bifocals (1784)
Phonograph (1784)

{ Title = Bifocals, YearOfPublication = 1784 }
{ Title = Phonograph, YearOfPublication = 1877 }

{ Title = Bifocals, Year = 1784 }

Anonymous types are purely a C# feature, not a new kind of type in the runtime. When the compiler encounters the anonymous type syntax, it generates a CIL class with properties corresponding to the named values and data types in the anonymous type declaration.

Selecting into Anonymous Types with LINQ

Lastly, capitalizing on anonymous types, we could create an IEnumerable<T> collection where T is an anonymous type (see Listing 15.27 and Output 15.14).

Listing 15.27: Projection to an Anonymous Type

Click here to view code image

// ...
IEnumerable<string> fileList = Directory.EnumerateFiles(
    rootDirectory, searchPattern);
var items = fileList.Select(                                                        
    file =>                                                                         
    {                                                                               
        FileInfo fileInfo = new FileInfo(file);                                     
        return new                                                                  
        {                                                                           
            FileName = fileInfo.Name,                                               
            Size = fileInfo.Length                                                  
        };                                                                          
    });
// ...

Output 15.14

Click here to view code image

{ FileName = AssemblyInfo.cs, Size = 1704 }
{ FileName = CodeAnalysisRules.xml, Size = 735 }
{ FileName = CustomDictionary.xml, Size = 199 }
{ FileName = EssentialCSharp.sln, Size = 40415 }
{ FileName = EssentialCSharp.suo, Size = 454656 }
{ FileName = EssentialCSharp.vsmdi, Size = 499 }
{ FileName = EssentialCSharp.vssscc, Size = 256 }
{ FileName = intelliTechture.ConsoleTester.dll, Size = 24576 }
{ FileName = intelliTechture.ConsoleTester.pdb, Size = 30208 }

The output of an anonymous type automatically shows the property names and their values as part of the generated ToString() method associated with the anonymous type.

Projection using the Select() method is very powerful. We already saw how to filter a collection vertically (reducing the number of items in the collection) using the Where() standard query operator. Now, by using the Select() standard query operator, we can also reduce the collection horizontally (making fewer columns) or transform the data entirely. By adding support of anonymous types, we can Select() an arbitrary “object” by extracting only those pieces of the original collection that are desirable for the current algorithm but without having to declare a class to contain them.

More about Anonymous Types and Implicit Local Variables

In Listing 15.26, member names for the anonymous types are explicitly identified using the assignment of the value to the name for patent1 and patent2 (e.g., Title = "Phonograph"). However, if the value assigned is a property or field call, the name may default to the name of the field or property rather than explicitly specifying the value. For example, patent3 is defined using a property named Title rather than an assignment to an explicit name. As Output 15.13 shows, the resultant property name is determined, by the compiler, to match the property from where the value was retrieved.

Both patent1 and patent2 have the same property names with the same data types. Therefore, the C# compiler generates only one data type for these two anonymous declarations. In contrast, patent3 forces the compiler to create a second anonymous type because the property name for the patent year is different from that in patent1 and patent2. Furthermore, if the order of the properties were switched between patent1 and patent2, these two anonymous types would not be type-compatible. In other words, the requirements for two anonymous types to be type-compatible, within the same assembly are a match in property names, data types, and order of properties. If these criteria are met, the types are compatible even if they appear in different methods or classes. Listing 15.28 demonstrates the type incompatibilities.

Listing 15.28: Type Safety and Immutability of Anonymous Types

Click here to view code image

class Program
{
  static void Main()
  {
      var patent1 =
          new
          {
              Title = "Bifocals",                
              YearOfPublication = "1784"         
              YearOfPublication = "1877",        
              Title = "Phonograph"               
              Year = patent1.YearOfPublication   
      var worldCup2006Finalists = new[]          
      {                                          
          };
      var patent2 =
          new
          {
          };
      var patent3 =
          new
          {
              patent1.Title,
          };

      // ERROR: Cannot implicitly convert type
      //        'AnonymousType#1' to 'AnonymousType#2'
      patent1 = patent2;
      // ERROR: Cannot implicitly convert type
      //        'AnonymousType#3' to 'AnonymousType#2'
      patent1 = patent3;

      // ERROR: Property or indexer 'AnonymousType#1.Title'
      //        cannot be assigned to -- it is read-only
      patent1.Title = "Swiss Cheese";
  }
}

The first two compile-time errors assert that the types are not compatible, so they will not successfully convert from one to the other. The third compile-time error is caused by the reassignment of the Title property. Anonymous types are immutable, so it is a compile-time error to change a property on an anonymous type once it has been instantiated.

Although not shown in Listing 15.28, it is not possible to declare a method with an implicit data type parameter (var). Therefore, instances of anonymous types can be passed outside the method in which they are created in only two ways. First, if the method parameter is of type object, the anonymous type instance may be passed outside the method because the anonymous type will convert implicitly. A second way is to use method type inference, whereby the anonymous type instance is passed as a method type parameter that the compiler can successfully infer. Thus, calling void Method<T>(T parameter) using Function(patent1) would succeed, although the available operations on parameter within Function() are limited to those supported by object.

Although C# allows anonymous types such as the ones shown in Listing 15.26, it is generally not recommended that you define them in this way. Anonymous types provide critical functionality with C# 3.0 support for projections, such as joining/associating collections, as we discuss later in the chapter. Nevertheless, you should generally reserve anonymous type definitions for circumstances where they are required, such as aggregation of data from multiple types.

At the time that anonymous types were introduced, they were a breakthrough that solved an important problem: declaring a temporary type on the fly without the ceremony of having to declare a full type. Even so, they have several drawbacks, as detailed earlier. Fortunately, C# 7.0 tuples avoid these drawbacks and, in fact, essentially obviate the need for using anonymous types altogether. Specifically, tuples have the following advantages over anonymous types:

• Provide a named type that can be used anywhere a type can be used, including declarations and type parameters

• Available outside the method in which they are instantiated

• Avoid type “pollution” with types that are generated but rarely used

One way in which tuples differ from anonymous types is that anonymous types are reference types and tuples are value types. Whether this difference is advantageous to one approach or the other depends on the performance characteristics needed. If the tuple type is frequently copied and its memory footprint is more than 128 bits, a reference type is likely preferable. Otherwise, using a tuple will most likely be more performant—and a better choice to default to.