Chapter 6
Using Standard Query Operators
In This Chapter
Understanding How LINQ Is Implemented
Constructing a LINQ Query
Filtering Information
Using Quantifiers
Partitioning with Skip and Take
Using Generation Operations
Equality Testing
Obtaining Specific Elements from a Sequence
Appending Sequences with Concat
“Most people have the will to win; few have the will to prepare to win.;
—Bobby Knight
There are a lot of keywords that make up the Language INtegrated Query (LINQ) grammar. Many of these keywords are similar to the Structured Query Language (SQL) and some elements are different. In addition to keywords, there are capabilities that are represented both by keywords and extension methods and capabilities only available by blending extension methods with LINQ queries.
This chapter explores general query syntax, query operators, and extension methods, including filtering, quantifiers, partitioning, generation, conversions, concatenation, and sequence-equality testing. Refer to Chapter 2, “Using Compound Type Initialization,” for conversion operators, Chapter 7, “Sorting and Grouping Queries,” for information on sorting and grouping, Chapter 8, “Using Aggregate Operations,” for aggregate operators, Chapter 9, “Performing Set Operations,” for set operations, Chapter 10, “Mastering Select and SelectMany,” for projects with select, and Chapter 11, “Joining Query Results,” for detailed information on joins.
Understanding How LINQ Is Implemented
LINQ is an extension to .NET. The grammar is new and adds elements to languages like C# (and VB .NET). However, LINQ did not crop up like a weed in a suburban garden. LINQ queries are emitted as calls to extension methods and generic delegates and these are integral parts of the .NET Framework.
Chapters 1 through 5 methodically demonstrate how anonymous types, compound initialization, extension methods, generics, projections, Lambda Expressions, and generic delegates all play a role in supporting LINQ.
Constructing a LINQ Query
LINQ queries start with the keyword from. Several people have asked why we don’t begin queries with select—as it is, select comes at the end. The answer ostensibly is that because IntelliSense unfolds as you type the LINQ queries, the from clause has to be first.
Consider a projection. In the select clause, you can define new types and initialize named properties of those new types from elements of the source type. If the select were first, it would be difficult, if not impossible, to use features like named initialization to define the projections because the type you are projecting from would be as yet unknown. It’s a reasonable argument.
After you get past from-first and select-last, basic LINQ queries are pretty straightforward. Between the from clause and the select clause, you can find joins, wheres, order-bys, and intos.
This chapter provides examples demonstrating query operators. Several of the larger, more complicated LINQ constructs, such as joins and groups, are covered in the chapters of Part II.
Filtering Information
There are two kinds of filters: the traditional filter with the where clause, which works similarly to SQL queries, and the new filter OfType. OfType filters on a value’s ability to be cast to a specific type. Listing 6.1 contains a query that filters numbers that have a magnitude (absolute value) greater than 5, and Listing 6.2 contains a query that returns elements from an array that can be converted to an integer. (Figure 6.1 shows how the sequence is converted in Listing 6.1.)
Figure 6.1 The sequence of positive and negative integers is reduced to just those with an absolute value greater than five.
Listing 6.1 Return Elements from an Array That Has an Absolute Value Greater Than 5
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace FilteringData
{
class Program
{
static void Main(string[] args)
{
var numbers = new int[] { -1, -32, 3, 5, -8, 13, 7, -41 };
var magnitude = from n in numbers where Math.Abs(n) > 5 select n;
foreach (var m in magnitude)
Console.WriteLine(m);
Console.ReadLine();
}
}
}
Listing 6.2 Using the Extension Method OfType; There Is No LINQ Keyword for OfType
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace OfTypeFilterDemo
{
class Program
{
static void Main(string[] args)
{
var randomData = new object[]{1, “two”, 3, “four”, 5.5, “six”, 7M};
var canBeInt = from r in randomData.OfType<int>() select r;
foreach(var i in canBeInt)
Console.WriteLine(i);
Console.ReadLine();
}
}
}
The code in Listing 6.2 returns the values 1 and 3 (see Figure 6.2). Although you can convert 7M (a decimal number) to an integer with the Convert class and ToInt32 method, OfType does not perform that conversion. You can learn more about conversion operators in Chapter 2.
Figure 6.2 OfType can convert types in a sequence but not as thoroughly as the Convert class.
Using Quantifiers
The quantifiers All, Any, and Contains are extension methods that return a Boolean indicating whether some or all of the elements in a collection satisfy the argument condition of the quantifier. All determines if every member satisfies the condition. Any returns a Boolean indicating whether any one or more members satisfy a condition, and Contains looks for a single element matching the condition.
Listing 6.3 demonstrates the quantifiers All and Any. The first check uses Any and a Lambda Expression to determine whether the species of any “family member” is feline, and the second check tests to determine if every family member is a dog (Species == “Canine”).
Listing 6.3 Demonstrating Quantifiers Any and All
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace QuantifierDemo
{
class Program
{
static void Main(string[] args)
{
var household = new List<FamilyMember>{
new FamilyMember{Name=“Paul Kimmel”, Species=“Lunkhead in Chief”,
Gender=“Male”},
new FamilyMember{Name=“Dena Swanson”, Species=“Boss”, Gender=“Female”},
new FamilyMember{Name=“Alex Kimmel”, Species=“Princess”, Gender=“Female”},
new FamilyMember{Name=“Noah Kimmel”, Species=“Annoying Brother”,
Gender=“Male”},
new FamilyMember{Name=“Joe Swanson”, Species=“Homosapien”, Gender=“Male”},
new FamilyMember{Name=“Ruby”, Species=“Canine”, Gender=“Female”},
new FamilyMember{Name=“Leda”, Species=“Canine”, Gender=“Female”},
new FamilyMember{Name=“Big Mama”, Species=“Feline”, Gender=“Female”}
};
bool anyCats = household.Any(m⇒m.Species == “Feline”);
Console.WriteLine(“Do you have cats? {0}”, anyCats);
bool allDogs = household.All(m⇒m.Species == “Canine”);
Console.WriteLine(“All gone to the dogs? {0}”, allDogs);
Console.ReadLine();
}
}
public class FamilyMember
{
public string Name{get; set; }
public string Species{ get; set; }
public string Gender{ get; set; }
public override string ToString()
{
return string.Format(“Name={0}, Species={1}, Gender={2}”,
Name, Species, Gender);
}
}
}
Notice that the class FamilyMember uses the automatic properties feature of .NET. Automatic properties are properties introduced with the normal property syntax—everything except the getter and setter methods. Use this technique for properties that do nothing more than return the underlying field or set the value of the underlying field. (Refer to Chapter 5, “Understanding Lambda Expressions and Closures,” for more on automatic properties.)
The Contains extension method accepts an instance of the type contained in the collection and tests to determine if the collection contains that particular member. You can pass a second argument that implements IEqualityComparer. The contract for IEqualityComparers is that a class that implements IEqualityComparer must provide an implementation for Equals and GetHashCode.
Partitioning with Skip and Take
Skip, SkipWhile, Take, and TakeWhile are used to partition collections into two parts and then return one of the parts. These partition features are only implemented as extension methods. Skip jumps to the indicated argument position in the sequence and returns everything after that. SkipWhile accepts a predicate and instead of using a position, it skips until the predicate evaluates to false. Take grabs all of the elements up to the specified position in the index, and TakeWhile grabs all of the items in a sequence as long as the predicate evaluates to true.
Listing 6.4 demonstrates Skip and Take. Take(5) returns a sequence containing the first five elements, the numbers 1 through 5, and Skip(3) returns the sequence containing the numbers 4 through 10. TakeWhile and SkipWhile substitute a test condition instead of using a cardinal value. For example, ints.TakeWhile(n⇒n<=5) yields the same result as Take(5). (See Figure 6.3 for the output visually depicted.)
Figure 6.3 Take(5) and Skip(3) yield the sequences as depicted in this figure.
Listing 6.4 Partitioning with Skip and Take
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace SkipDemo
{
class Program
{
static void Main(string[] args)
{
var ints = new int[]{1,2,3,4,5,6,7,8,9,10};
var result1 = ints.Take(5);
var result2 = ints.Skip(3);
Array.ForEach(result1.ToArray(), ⇒ Console.WriteLine(n));
Console.ReadLine();
Array.ForEach(result2.ToArray(), n⇒ Console.WriteLine(n));
Console.ReadLine();
}
}
}
Using Generation Operations
The generation operations are DefaultIfEmpty, Empty, Range, and Repeat. These are implemented as extension methods. Generation operations are useful for creating new sequences of values. For example, Empty creates an empty collection of a given type. The brief descriptions and examples (or references to examples already provided) follow.
DefaultIfEmpty
The following brief fragment purposely creates an empty array to demonstrate DefaultIfEmpty. In the fragment, DefaultIfEmpty creates a one-element collection with the default value of 0 for the contained type—a char.
var empties = Enumerable.DefaultIfEmpty<char>(Enumerable.Empty<char>());
Console.WriteLine(“Count: {0}”, empties.Count());
Console.ReadLine();
For another DefaultIfEmpty example, see the section on left joins in Chapter 11.
Empty
As introduced in the previous subsection (and also demonstrated in Chapter 8), Empty creates an empty collection of the type expressed by the parameter. For example, Enumerable.Empty<int> creates an empty collection of integer types.
Range
The Range extension method fills a new sequence of numbers. The arguments are start and count. For example, Enumerable.Range(0, 10) creates a sequence starting with 0 and containing 10 integers—0, 1, 2, 3, 4, 5, 6, 7, 8, 9. Here is a fragment that creates a range of numbers from 10 to 109 and uses the Array.ForEach method and an implicit generic Action delegate to display the sequence.
var sequence = Enumerable.Range(10/*start*/, 100/*count*/);
Array.ForEach(sequence.ToArray(), n⇒Console.WriteLine(n));
Console.ReadLine();
Repeat
If you want to fill a sequence with a repeated value, use the Repeat extension method. (These generation methods are useful for producing simple test values.) The following fragment puts one hundred occurrences of the word “test” in a sequence. (The extra two lines let you explore the sequence; you can plug this into any sample console application to test the code.)
var someStrings = Enumerable.Repeat(“test”, 100);
Array.ForEach(someStrings.ToArray(), s⇒Console.WriteLine(s));
Console.ReadLine();
Listing 6.5 uses compound named type initialization and Repeat to generate some test objects. (This beats what we usually have to do—especially authors—copy and paste data changing random parts of the data, and it might be useful for testers, too.)
Listing 6.5 Using Repeat and Compound Named Type Initialization to Create Some Test Data
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace RangeDemo2
{
class Program
{
public class Customer
{
public int ID{ get; set; }
public string CompanyName{ get; set; }
public string City{ get; set; }
}
static void Main(string[] args)
{
Action<Customer> print = c ⇒
Console.WriteLine(“ID: {0}
Customer: {1}
City: {2}
”,
c.ID, c.CompanyName, c.City);
var customers =
Enumerable.Repeat<Customer>(new Customer{
ID=DateTime.Now.Millisecond,
CompanyName=“Test Company”,
City=“Test City”}, 100);
Array.ForEach(customers.ToArray(), c⇒print(c));
Console.ReadLine();
}
}
}
This example has only one hundred duplicate Customer objects, but for testing purposes this might be sufficient.
Equality Testing
There was a time in the not-too-distant past that you had to know a lot about a computer’s BIOS (basic input/output system) and interrupts and interrupt handlers to perform everyday tasks. (Peter Norton was one of the people who led that education charge.) For example, interrupt 0x21 and functions 0x4E and 0x4F performed the task of finding the first and subsequent files based on a file mask. Knowledge of interrupts wasn’t just for assembly language programmers either. Writing interrupt handlers 15 years ago was as common as writing to the Windows application programming interface (API) was just 5 or 10 years ago.
These capabilities—these BIOS services—are still alive and well, but they are tidily bundled up under the .NET Framework and underneath that the Windows API. (This is progress.) Now, you can read all of the folders on a path and with LINQ compare and figure out what is different with four method calls: Directory.GetDirectories (twice), Enumerator.SequenceEqual, and Enumerator.Except (set-difference). (And, Peter Norton—a hero to some early PC geeks—is living comfortably in Santa Monica off the proceeds from the sale of Norton Utilities.)
Listing 6.6 demonstrates the SequenceEqual extension method. This method compares a source and target sequence and returns a Boolean indicating whether they contain the same elements.
Listing 6.6 SequenceEqual Is Used Here to Determine if Two Paths Contain the Same Folders and the Set-Difference Method Except Indicates What’s Missing
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
namespace SequenceEqualDemo
{
class Program
{
static void Main(string[] args)
{
var folder1 = Directory.GetDirectories(“C:\”);
var folder2 = Directory.GetDirectories(“C:\TEMP”);
// are they equal
Console.WriteLine(“Equal: “ + folder1.SequenceEqual(folder2));
Console.ReadLine();
// What’s the difference?
var missingFolders = folder1.Except(folder2);
Console.WriteLine(“Missing Folders: {0}”, missingFolders.Count());
Array.ForEach(missingFolders.ToArray(),
folder⇒Console.WriteLine(folder));
Console.ReadLine();
}
}
}
For more on set operations, refer to Chapter 9.
Obtaining Specific Elements from a Sequence
The element operations are all implemented as extension methods without LINQ keywords. The element methods are: ElementAt, ElementAtOrDefault, First, FirstOrDefault, Last, LastOrDefault, Single, and SingleOrDefault. Each in its way is designed to return a single element from a sequence.
ElementAt returns a single element from a sequence at the specified index. ElementAtOrDefault returns an element at the index or a default value if the element is out of range. First returns the first element of a sequence, and FirstOrDefault returns the first element of a sequence matching the criteria or a default value. Last and LastOrDefault methods act like First and FirstOrDefault but act on the last element. Single returns the only element if there is just one, and SingleOrDefault returns the only element that satisfies a condition or a default value. Listing 6.7 demonstrates each of these extension methods.
Listing 6.7 Demonstrating Element Operators
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.Data.SqlClient;
namespace ElementDemo
{
class Program
{
static void Main(string[] args)
{
var numbers = new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9};
// first
var first = (from n in numbers select n).First();
Console.WriteLine(“First: {0}”, first);
var firstOrDefault = (from n in numbers select n).FirstOrDefault(
n⇒n>10);
Console.WriteLine(“First or default: {0}”, firstOrDefault);
// last
var last = (from n in numbers select n).Last();
Console.WriteLine(“Last: {0}”, last);
var lastOrDefault = (from n in numbers select n).LastOrDefault(
n⇒n<5);
Console.WriteLine(“Last or default: {0}”, lastOrDefault);
// single
var single = (from n in numbers where n==5 select n).Single();
Console.WriteLine(“Single: {0}”, single);
var singleOrDefault = (from n in numbers select n).SingleOrDefault(
n⇒n==3);
Console.WriteLine(“Single or default: {0}”, singleOrDefault);
// element at
var element = (from n in numbers where n < 8 select n).ElementAt(6);
Console.WriteLine(“Element at 6: {0}”, element);
var elementOrDefault = (from n in numbers select n).ElementAtOrDefault(11);
Console.WriteLine(“Element at 11 or default: {0}”, elementOrDefault);
Console.ReadLine();
}
}
}
Appending Sequences with Concat
Concat is an extension method that concatenates one sequence onto a second. There are no LINQ keywords for concatenation. You can use Concat on an explicit sequence, providing the second sequence as the argument to Concat, or you can invoke Concat on an implicit sequence at the end of a query, as shown in Listing 6.7 with the element operators. Listing 6.8 demonstrates concatenating two sequences of numbers together. (See Figure 6.4 for the visualization.)
Listing 6.8 Concatenating Sequences Together with the Concat Extension Method
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.Data.SqlClient;
namespace ElementDemo
{
class Program
{
static void Main(string[] args)
{
var numbers = new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9};
var moreNumbers = new int[]{10, 11, 12, 13, 14, 15};
foreach( var n in numbers.Concat(moreNumbers))
Console.WriteLine(n);
Console.ReadLine();
}
}
}
Figure 6.4 Concatenation appends two sequences to make a new sequence containing all elements.
Summary
This chapter introduced some of the standard query operators that make up part of the full capability of the .NET Framework. Many of these are implemented only as extension methods, but can be easily integrated alongside LINQ using simple method invocation syntax. Included in this group are filtering, quantifiers, partitioning, generation, equality testing, element operations, and concatenation.
There are additional and substantial LINQ keywords and query operations that are covered in detail in Part II (which makes up the next six chapters).