Have you wondered if and when you should use the new LINQ features in .Net 3.5?

Like, where should I put a new extension method? Should I use Func<T> or a custom delegate? How do I best implement a mix-in (extension methods on an interface)?

Well, Mircea Trofin has just published a new draft of some LINQ design guidelines. You might just find your answers there.

When learning LINQ-to-SQL, it’s not immediately obvious how to do an update. Querying is easy, and there are methods for inserting and deleting. Updating usually occurs by modifying an object already known to the DataContext and then calling SubmitChanges on the context.

var product = (from p in dataContext.Products
               where p.ProductID == 1
               select p).Single();

product.Name = "Richard's product";

dataContext.SubmitChanges();

It’s nice to see that MSDN documentation actually addresses the obvious arising question:

Q. Can I update table data without first querying the database?

A. Although LINQ to SQL does not have set-based update commands, you can use either of the following techniques to update without first querying:

• Use ExecuteCommand to send SQL code.
• Create a new instance of the object and initialize all the current values (fields) that affect the update. Then attach the object to the DataContext by using Attach and modify the field you want to change.

If you ask me, using an ExecuteCommand defeats the object of LINQ-to-SQL. After all, we’re using it as a data-access-layer to generate entities in code based on the data schema. The last thing we want to have to do is start writing strings of SQL.

So that’s out.

I’m So Attached

The second option is to use the Attach method on the table we’re updating. I’ve tried to use the Attach method before, but it didn’t work.

To see what I mean, try this in LINQPad (using the AdventureWorks database from CodePlex):

var product = new Product();
product.ProductID = 1;
product.Name = "Richard's product";

Products.Attach(product);

var changeSet = GetChangeSet();

changeSet.Dump();

You’ll see that there are no changes to make, so no updates are made.

Now, move the line of code that does the update after the call to Attach, so as to attach the product first before updating the field:

var product = new Product();
product.ProductID = 1;

Products.Attach(product);

//Notice we're doing this laterproduct.Name = "Richard's product";

var changeSet = GetChangeSet();

changeSet.Dump();

The changeset now shows that one update is ready to be made. Unfortunately, if you call SubmitChanges, you get an exception:

SqlDateTime overflow. Must be between 1/1/1753 12:00:00 AM and 12/31/9999 11:59:59 PM.

The MSDN documentation isn’t so clear here. What do we need to set to get it to work?

The SQL being sent to the database is instructive:

UPDATE [Production].[Product]
SET [Name] = @p9
WHERE ([ProductID] = @p0) AND ([Name] IS NULL)
AND ([ProductNumber] IS NULL)
AND (NOT ([MakeFlag] = 1))
AND (NOT ([FinishedGoodsFlag] = 1))
AND ([Color] IS NULL)
AND ([SafetyStockLevel] = @p1)
AND ([ReorderPoint] = @p2)
AND ([StandardCost] = @p3)
AND ([ListPrice] = @p4)
AND ([Size] IS NULL)
AND ([SizeUnitMeasureCode] IS NULL)
AND ([WeightUnitMeasureCode] IS NULL)
AND ([Weight] IS NULL)
AND ([DaysToManufacture] = @p5)
AND ([ProductLine] IS NULL)
AND ([Class] IS NULL)
AND ([Style] IS NULL)
AND ([ProductSubcategoryID] IS NULL)
AND ([ProductModelID] IS NULL)
AND ([SellStartDate] = @p6)
AND ([SellEndDate] IS NULL)
AND ([DiscontinuedDate] IS NULL)
AND ([rowguid] = @p7)
AND ([ModifiedDate] = @p8)
-- @p0: Input Int (Size = 0; Prec = 0; Scale = 0) [1]
-- @p1: Input SmallInt (Size = 0; Prec = 0; Scale = 0) [0]
-- @p2: Input SmallInt (Size = 0; Prec = 0; Scale = 0) [0]
-- @p3: Input Decimal (Size = 0; Prec = 29; Scale = 4) [0]
-- @p4: Input Decimal (Size = 0; Prec = 29; Scale = 4) [0]
-- @p5: Input Int (Size = 0; Prec = 0; Scale = 0) [0]

The problem is now that LINQ-to-SQL tables automatically use optimistic concurrency, and for some reason (maybe only in LINQPad) the SellStartDate is not set, and neither are the rowguid or ModifiedDate fields. In any case, either we have to turn off optimistic concurrency, or set all the field values ourselves.

Using Optimistic Concurrency with Updates

Unfortunately, by default all fields are used for optimistic concurrency, so you have to set values for all fields. If you were using ASP.Net you could keep a copy of the variables in the session state or something, but at least it saves you a round trip to the database.

Here’s the code that actually does perform an update:

var product = new Product();
product.ProductID = 1;
product.Name = "Adjustable Race";
product.ProductNumber = "AR-5381";
product.MakeFlag = false;
product.FinishedGoodsFlag = false;
product.Color = null;
product.SafetyStockLevel = 1000;
product.ReorderPoint = 750;
product.StandardCost = 0.0000M;
product.ListPrice = 0.0000M;
product.Size = null;
product.SizeUnitMeasureCode = null;
product.WeightUnitMeasureCode = null;
product.Weight = null;
product.DaysToManufacture = 0;
product.ProductLine = null;
product.Class = null;
product.Style = null;
product.ProductSubcategoryID = null;
product.ProductModelID = null;
product.SellStartDate = DateTime.Parse("6/1/1998 12:00:00 AM");
product.SellEndDate = null;
product.DiscontinuedDate = null;
product.Rowguid = new Guid("694215b7-08f7-4c0d-acb1-d734ba44c0c8");
product.ModifiedDate = DateTime.Parse("3/11/2004 10:01:36.827 AM");

Products.Attach(product);

// Make the change here
product.Name = "Richard's product";

SubmitChanges();

So you can avoid two trips to the database by doing the update manually and using the Attach method on the table you are updating.

Mind you, I’m not sure that’s worth it.

If you want to know more about updating LINQ-to-SQL objects with the DataContext, I recommend Pro LINQ: Language Integrated Query in C# 2008. It’s got 6 chapters just about LINQ-to-SQL, the DataContext, and concurrency issues.

P.S. In case you’re interested, I generated some of that code in LINQPad using the following code (it doesn’t work for dates or Guids, before anyone complains):

var product = (from p inProducts

wherep.ProductID == 1

selectp).Single();

var fields = from field inproduct.GetType().GetFields()

select new{

Name = field.Name,

Value = field.GetValue(product),

TypeName = field.FieldType.Name };

foreach (var field infields)

Console.WriteLine("product.{0} = {1};", field.Name,

field.TypeName == "String"? "\""+

field.Value + "\"":

field.Value ?? "null");

Quick tip: If you want to see the SQL generated by LINQ to SQL for a query or command, simply set the Log property of your generated DataContext class to an instance of a TextReader.

If this is your code:

using System;
using System.Linq;
using System.Data.Linq;

namespace LINQtoSQLConsole {
  class Program {
    static void Main(string[] args) {

      var db = new NorthwindDataContext();

      // Use the console to see the SQL
      db.Log = Console.Out;

      // A query
      var cust = db.Customers.Single(
                    c => c.CustomerID == "ALFKI");

      // An update
      cust.Region = "Northwest";
      db.SubmitChanges();
    }
  }
}

… then this is what you’ll see:

Pretty good, eh?

When you started using LINQ, did you think it looked like SQL? I did.

The more I learned LINQ, the more I realized it wasn’t anything like SQL. Take grouping, for example. Because LINQ has a group by statement, and it looks like SQL, I assumed that the syntax for grouping in LINQ would be just like SQL. Ha ha! Wrong! As soon as I tried to use it, I discovered that the LINQ syntax is not only nothing like the SQL equivalent, but the whole grouping concept in LINQ is completely different too.

At first glance, the two syntaxes look slightly similar.

SQL:

select ReportsTo, count(LastName) as NameCount
from Employees
group by ReportsTo

LINQ (C#):

from employee in Employees
group employee by employee.ReportsTo

Ignoring the omission of the select statement from LINQ, and the requirement of a range variable, they do look similar. But looks can be deceiving.

Let me explain why.

Grouping in SQL

In SQL, you typically group a set of entities using the following syntax:

select ReportsTo, count(LastName) as NameCount
from Employees
group by ReportsTo

This kind of query results in a simple table with two columns: ReportsTo and NameCount. The count of the names was grouped into a single column using an aggregate function.

Of course, this is a simple example; you could do much more complex grouping by using more expressions in the aggregation or the group by statement. Whatever the grouping though, the result is a flat table.

LINQ is different. Completely different.

Grouping in LINQ

Before you can understand the LINQ syntax, you have to understand how grouping works in LINQ.

When you use the group by statement, the C# compiler translates it to the LINQ Enumerable’s GroupBy extension method. The GroupBy method returns an object of type

  IEnumerable<IGrouping<TKey, TSource>>

The GroupBy method takes a parameter of a delegate which specifies the key of each grouping. The key is defined to be of type TKey, and is usually inferred by the compiler depending on the expression.

The key to understanding the C# query syntax is that you don’t get back a flat structure from group by, as you would with SQL. You actually get an “IEnumerable” of “IGrouping” objects.

The IGrouping interface has only one property, “Key”, and implements IEnumerable<TElement>. Again, TElement is usually inferred. “Key” is typed as the generic type you define for TKey. For the minute, that’s not so important, so we’ll come back to the Key later.

One of the first indications that LINQ is different to SQL is that you discover it is possible to finish a LINQ statement with a group by statement without needing a Select. As IGrouping implements IEnumerable, group by actually returns an IEnumerable of IEnumerable. The result is a set of sets. It’s not a flat table. To use it, you have to iterate over it again or perform another operation on the set.

Let’s see if an example can make it clearer.

When grouping a set of employees, you might use the following code:

var emp1 = new { ReportsTo = 1, LastName = "Richard" };
var emp2 = new { ReportsTo = 1, LastName = "Christopher" };
var emp3 = new { ReportsTo = 2, LastName = "John" };
var emp4 = new { ReportsTo = 2, LastName = "Greg" };
var employees = new[] { emp1, emp2, emp3, emp4 };

var groups =
  from employee in employees
  group employee by employee.ReportsTo;

foreach (var group in groups)
  Console.WriteLine("{0} employees report to {1}.",
    group.Key, group.Count());

(Don’t worry about the anonymous class definitions for now. You just have to know that my employee objects are all treated as the same type internally, so this is valid C# code.)

The employees variable is of type IEnumerable, so it can be used in a LINQ statement. The groups variable is also an IEnumerable, as it is a set of results, as usually returned by a LINQ statement. Remember though, it’s not like the set you would get back in a SQL statement. It is actually an IEnumerable of IGrouping, or an IEnumerable of IEnumerable.

To do something with the grouping, you have to either treat it as another IEnumerable, and iterate over it, or use methods like Count() to do something with it.

foreach (var group in groups)
  foreach (var employee in group)
    Console.WriteLine("{0} reports to {1}",
      employee.LastName, employee.ReportsTo);

This is how LINQPad shows what is contained in the groups variable:

Notice that each IGrouping has an associated Key property of type Int32.

Grouping Syntax in LINQ

So now it should be a little easier to understand how to use the LINQ group by statement.

In LINQ, you always need to declare the range variable between group and by.

from employee in Employees

group employee by employee.ReportsTo

After by, you give an expression which returns the value for the Key. The enumeration is then split into groups of IGroupings where the key for each item in that IGrouping is the same.

So here, we specify the employee.ReportsTo variable as the key. Each employee with a different value for ReportsTo creates a new IGrouping with a Key of that value. The employee is then added to that new IGrouping as an item. The IGrouping is then added to the returned IEnumerable<IGrouping<>>.

The next time an item in the employees variable has the same value for ReportsTo as the Key property of an already-existing IGrouping object, the item is simply added to that IGrouping.

Of course, that’s probably not the way it works under the covers. But I’m not trying to specify the algorithm, just what happens in the end result.

In the end, an object implementing IEnumerable<IGrouping<>> is returned by the LINQ statement. As each item in the IEnumerable is an IGrouping, you can either use it’s Key or treat it as an IEnumerable.

So there you have it. Just remember, group by is nothing like SQL, and it returns an IEnumerable of IGrouping, and you should stay happy.

At least until you meet SelectMany… :=)

If you want to learn more about grouping in LINQ, take a look at chapter 9 of the excellent C# 3.0 in a Nutshell: A Desktop Quick Reference (In a Nutshell (O’Reilly)), by Ben and Joseph Albahari. The chapter has some great explanations and examples. In particular, they have an example of grouping with multiple keys, which I found very interesting.

Last week I had to prepare a small presentation for a new LINQ workshop. For my research, I browsed through the MSDN documentation on LINQ where I came across a reference to "LINQ to DataSets". I couldn’t find any more information about it though, and so LINQ to DataSets got a small mention in my presentation, but I couldn’t say anything more about it.

This morning I discovered that Fabrice Marguerie and his co-authors have published a bonus 47-page chapter of their book "LINQ in Action", all about this very subject. The chapter is called "Working with LINQ and DataSets", and they have made a PDF version available online for FREE download from the Manning Press website.

You can go get it right now.

Thanks Fabrice, for filling in the missing gap.

I’m going back to update my presentation…

I am really getting into LINQ now! I think it’s fantastic. I recently wanted to develop a quick drop-down list in ASP.Net which allows a user to select a time of day from a list. The times are 15 minutes apart, so the list would look like this:

…
08:00
08:15
08:30
08:45
09:00
…

… and so on.

Before LINQ, I would have done this with a for loop, like this:

List<string> times = new List<string>();
for (int hour = 0; hour < 24; hour++)
  for (int minute = 0; minute < 60; minute++)
    if (minute % 15 == 0)
       times.Add(string.Format(
         "{0:00}:{1:00}", hour, minute));

That’s not difficult, although it’s not so easy to understand. I would have to write a small console app or test to make sure I had done it correct though.

I thought this might be a good opportunity to use LinqPad. It’s a great tool. You can use it to test a LINQ statement in a live window, so I thought I’d give it a go.

First I needed a LINQ statement to test.

Designing the LINQ Statement

The first thing I needed was an integer for the hour. That’s quite easy, especially if you use the Range Extension Method technique as I wrote about before. That method allows you to create an IEnumerable<int> from a simple statement, like 1.To(10). So to get the hours of the day, I can simply start with:

from hour in 0.To(23)

Now I need to do a cross-join with the minutes. In C# comprehension syntax, you do that by simply adding another from statement. At first, I thought I could do it by using an enumeration of minutes, specifying each value I wanted specifically. To do that I wrote this:

from hour in 0.To(23)
from minute in new[] {0, 15, 30, 45}

That works very nicely. I think it’s easy to understand too. Even if you were going to come back to it years from now, you could still easily see what is being done.

Another alternative is to do the same as the for statement above, go through the numbers 0 to 59, selecting only those which are divisible by 15. You would do that using another range and a filter:

from hour in 0.To(23)
from minute in 0.To(59)
where minute % 15 == 0

Again, I like the way this looks. Another developer should have no difficulty reading that.

To select the string, we will also use the same format as before, this time using the select keyword:

select string.Format("{0:00}:{1:00}", hour, minute)

You then simply assign the value of this statement to a variable, like this:

var times = from hour in 0.To(23)
            from minute in 0.To(59)
            where minute % 15 == 0
            select string.Format("{0:00}:{1:00}", hour, minute);

Using LinqPad to Test the Result

Now, if you haven’t already, go download LinqPad from here, and open it up. You will see a window to write a statement in the top right.

Before you continue, you need to add the extension methods to use my “To()” method. As a treat, just for you, here’s a simple assembly and a code file you can use straight away. Download it, save the binary dll somewhere, then do the following in LinqPad:

1. From the menu, click “Query”, then “Advanced Properties” (or press F4).
2. On the Additional References tab, click the “Add…” button.
3. Click the “Browse…” button on the window which pops up.
4. Go to the location you saved RichardBushnell.Extensions.dll to, and select it. You should see it added to the list of additional references.
5. Change to the Additional Namespace Imports tab.
6. Type “RichardBushnell.Extensions”
7. Click “OK”.

Now, put your cursor in the Query1 tab, and paste in the following:

from hour in 0.To(23)
from minute in 0.To(59)
where minute % 15 == 0
select string.Format("{0:00}:{1:00}", hour, minute)

Press F5, and you will see a generated list like this:

And that’s it! You can now see exactly what you will get when you run this code.

What do you think? I’d love to hear your comments.