The Language-Integrated Query (LINQ) framework is what mostly changes the programming technique in the .NET world. LINQ is a framework that lives within the .NET language and helps us create complex queries against any data source, such as in-memory objects using the LINQ to object-provider or against an Entity Framework database using the LINQ to entities provider.

Almost anything that is enumerable in the world has its own provider within a simplified index, as shown at the following link:

http://blogs.msdn.com/b/charlie/archive/2008/02/28/link-to-everything-a-list-of-linq-providers.aspx

Any iteration for the for or foreach statements is now made with LINQ. The same is made against any relational database accessed by an O/RM or any non-relational database too. LINQ has its own pseudo SQL language that is accessible within any .NET language. Here is a LINQ statement example:

var query = from i in items
            where i >= 100
            orderby i descending
            select i;

By analyzing the preceding statement, you should understand that it represents a simple query that will make a filter for in-memory data, then will order the filtered data, and later will output the data. The key concept of LINQ is that LINQ creates queries. It does not executes such queries; it simply declares queries.

Any query has a proper type. The type of query mentioned earlier is IOrderedEnumerable<int>, which means that it is an ordered enumerable collection. Because of the verbosity of the query's result type, in conjunction with the frequent usage of anonymous types, the var keyword is widely used when typing a variable that will contain the query itself, as the previous example showed.

Please remember that anonymous types will be visible to Intellisense (Visual Studio's suggestion system of the text editor) only within the code block in which it was created. This means that outside the method where we created the anonymous type, we will lose the Intellisense support. The anonymous type will be visible only to the assembly where it was created.

This means that if we need to use an anonymous typed object outside our assembly, we should instead use a statically typed type (not anonymous), or we need to use the Reflection namespace to read the internal marked properties that compose our anonymous type.

Back to LINQ, the query itself will be executed in a lazy fashion only when iterated by any foreach statement or when using specific Extension methods that will materialize ( produce results) the query in the ToArray or ToList methods:

var concreteValues = query.ToArray();

LINQ has a lot of extension methods to access its huge library. Actually, the preceding syntax is very limited compared to all the extension methods available from any enumerable collection. Here's an example:

var values = items
    .Where(i => i >= 100)
    .OrderByDescending(i => i)
    .ToArray();

All LINQ methods will work in a fluent way, appending any new altering method to the previous one. The Lambda expression is ubiquitous and is used to define new values, value mapping, filter predicates, and so on, so its syntactical deep knowledge is mandatory when using LINQ. The two examples provide identical results, although the second one will not store the query itself but only its concrete result.

The magic of LINQ is the ability to work in any application and query any kind of data, from variables to XMLs with a single syntax. Of course, some limitations are present when dealing with external resources such as databases, because some data providers cannot handle all LINQ methods. In such a case, an exception will be thrown by the provider itself. In modern .NET programming, any iterating logic is usually executed within a LINQ statement.

Another magic aspect of LINQ is the ability to return queries and not always data. This adds the ability to create new features without having to retrieve the same data more than once, or without having to write complex if/else statements with possible copy/paste programming. The following example shows how to split the query similar to what we have already seen in the previous example:

var query1= items as IEnumerable<int>;
var query2= query1.Where(i => i >= 100);
var query3 = query2.OrderByDescending(i => i);
var values = query3.ToArray();

The following example shows how to add new filters to the already filtered query. The two where statements will execute just like an and clause.

var query = items.Where(i => i >= 100);

if (SomeLogic())
    //another where is added to the LINQ
    query = query.Where(i => i <= 900);

Please note that using this query result can trigger an undesirably query materialization (execution) multiple times. If you want to work in values, it is always easier to materialize the LINQ with an ending that consists of the ToArray or ToList method.

LINQ offers, by default, any set operation such as Union, Distinct, Intersect, and Except; any aggregate operation such as Count, Sum, Max, Min, and Average; and any relational data operation such as Join, GroupJoin, and so on.

Transformation methods such as Select or SelectMany are also interesting because they give us the ability to change the object that flows from one LINQ step to the other letting us add/remove/change data. By the way, one of the greatest features of LINQ is the ability to append LINQ queries to another LINQ query, including when dealing with multiple LINQ data providers.

Here's an overview of LINQ's features:

//a dataset
var items = Enumerable.Range(1, 1000);

//a simple filter
var filter1 = items.Where(i => i <= 100);

//takes until matches a specific predicate
var filter2 = items.TakeWhile(i => i <= 100); //same as Where above.

//shape the original data item in another one
var shape1 = items.Select(x => new { Value = items });

//shape multi-dimensional data in flattened data
//this will produce a simple array from 1 to 9
var shape2 = new[] { new[] { 1, 2, 3 }, new[] { 4, 5, 6 }, new[] {7, 8, 9 } }.SelectMany(i => i);

//group data by divisible by 2
var group1 = items.GroupBy(i => i % 2 == 0);

//take only x values
var take1 = items.Take(10);

//take only after skipped x values
var skip1 = items.Skip(10);

//paginate values by using Take and Skip together
var page3 = items.Skip(2 * 10).Take(10);

//join values
var invoices = new[]
{
    new {InvoiceID=10, Total=44.50},
    new {InvoiceID=11, Total=34.50},
    new {InvoiceID=12, Total=74.50},
};
//join invoices with items array
//shape the result into a new object
var join1 = invoices.Join(items, i => i.InvoiceID, i => i, (a, b) => new { a.InvoiceID, a.Total, Index = b, });

All LINQ methods are combinable with any other data collection made by any other data provider. This means that we can actually make a join between two different database values, or between a database value and a file, or an XML, a Web Service, or a control on our UI, or anything else we could want. Obviously, things are not so easy when we want merge data from multiple LINQ data providers, especially because of the Entity Framework data provider that will try to translate anything written in LINQ into SQL. To help us avoid LINQ to SQL translation issues, there are techniques such as small materializations within the LINQ steps (like putting a ToArray() method before performing the join between different providers) or starting the query with an in-memory source instead of using an Entity Framework DbQuery class.

Entity Framework queries will be discussed later in Chapter 7, Database Querying. In the meantime, here is a simple example of a cross LINQ provider query:

var localDataset = new[]
{
    new { Latitude=41.88f, Longitude=12.50f, Location="Roma"},
    new { Latitude=45.46f, Longitude=9.18f, Location="Milano"},
    new { Latitude=59.32f, Longitude=18.08f, Location="Stockholm"},
};

//within the TestDB there is a simple table as
//CREATE TABLE [dbo].[Position] (
//[Latitude] [real] NOT NULL,
//[Longitude] [real] NOT NULL)
using (var db = new TestDBEntities())
{
    //this query starts from the local dataset
    //and later creates a join with the table within the database
    var query = from p in localDataset
                join l in db.Position on new { p.Latitude, p.Longitude } equals new { l.Latitude, l.Longitude }
                select new
                {
                    l.Latitude,
                    l.Longitude,
                    p.Location,
                };

    //materialize the query
    foreach (var position in query)
        Console.WriteLine("Lat {0:N2} Lon {1:N2}: {2}", position.Latitude, position.Longitude, position.Location);
}

Console.ReadLine();

By executing the example given, you will know how to join two different data providers by specifying the only coordinate couple that we want see in the database table.