This is the last of my posts on LINQ before switching over to Beta2; I'm playing catch-up but be aware that some things will have changed. I will post any changes for Beta 2 to both this and Being Ignorant in LINQ to SQL as time permits.

What are Specifications?

Specification is a design pattern authored by Martin Fowler and Eric Evans. If you are not familiar with specifications then you can read more about them in Eric Evans book Domain Driven Design, or just read this comprehensive paper. If you are not sure what Specifications are, you might want to read that before you read this. For those of you who just want a refresher, or who do not want to read the article first: a specification is a pattern for expressing a rule which you want to use to test an object. A specification is ultimately used to separate two orthogonal concerns: testing objects and the objects themselves. An example should make this clearer.

Instead of writing:

customer.IsInSalesRegion("London");

we want to write:

SalesRegionSpecification salesRegionSpecification = new SalesRegionSpecification("London");

salesRegionSpecification.IsSatisfiedBy(customer);

We separate the specification - the test for a customer being in London - from the customer itself. Separation of responsibility not only means that the code is easier to maintain, because changes to our specifications do not imply changes to Customer, but also allow us to extend the range of specifications without changing Customer. In addition as well as using specifications independently, we can combine them:

SalesRegionSpecification salesRegionSpecification = new SalesRegionSpecification("London");

AccountSinceSpecification accountSinceSpecification = new AccountSinceSpecification(new TimeSpan(1997,0,0));

Specification qualifyingLondonAccountsSpecification = salesRegionSpecification.And(accountSinceSpecification);

qualifyingLondonAccountsSpecification .IsSatisfiedBy(customer);

Note that a specification has characteristics shown by Fluent Interfaces in that it is readable API. The three primary uses of the Specialization pattern are to:

• Validate an object
• Select an object from a list
• Specify the requirements for creation of an object  

Hard Coded Specifications

The basic implementation of a specification is something like this:

public interface ISpecification
{
    bool IsSatisfiedBy(Customer customer);
}

public class CustomerInLondonSpecification : ISpecification
{
    public bool IsSatisfiedBy(Customer customer)
    {
        return customer.City == "London"
    }
}

and we can use it like this:

DataSource dataSource = new DataSource();
List<Customer> londonCustomers = dataSource.GetCustomerList().SelectSatisfying(new CustomerInLondonSpecification());

    ...

assuming something like: 

public static class Spec
{
   public static List<Customer> SelectSatisfying(this List<Customer> existingCustomers, ISpecification specification)
   {
       List<Customer> customers = new List<Customer>();
      foreach (Customer customer in existingCustomers)
      {
          if (specification.IsSatisfiedBy(customer))
              customers.Add(customer);
      }

      return customers;
}

Of course we can use delegates to do something similar by treating code as data, so we can call it at a time of our choosing. In C# 2.0 there is a even generic delegate for this purpose called Predicate: delegate Boolean Predicate<T>(T obj) and a number of methods, such as, on List<T>: List<T> FindAll(Predicate<T> match), which take a predicate to allow selection.

in C# 3.0 this enables us to use lambda expressions to write code as follows:

Predicate<Customer> isCustomerInLondon = c => c.City == "London"
List<Customer> londonCustomers = dataSource.GetCustomerList().FindAll(isCustomerInLondon);

or even

List<Customer> londonCustomers = dataSource.GetCustomerList().FindAll(c => c.City == "London");

which in essence, fulfills the implementation requirements of a hard coded specification. And of course LINQ really just gives us another way of writing this:

var query =
    from c in dataSource.GetCustomerList()
    where c.City == "London"
    select c;  

which shows the relationship between LINQ and specifications in that LINQ is one way of implementing selecting an object from a list.  

Parameterized Specifications

The trouble with hard-coded specifications is that we are going to end up with type explosion if we need a lot of them (or a lot of LINQ expressions), so we then arrive at the idea of parameterizing them. That way instead of a specification that checks if the customer is in London and one that checks if they are in Seattle, we can just have one to check if they are in a city and pass in the city that we are after as a paramter to the specification when we construct it.

So, extending the example above we could implement a parameterized method as follows:

public class CustomerForCitySpecification : ISpecification
{
    private string city;

    public CustomerForCitySpecification(string city)
    {
        this.city = city;
    }

    public bool IsSatisfiedBy(Customer customer)
    {
        return customer.City == city;
    }

}

 and use it like this:

ISpecification isInLondon = new CustomerForCitySpecification("London");
List<Customer> londonCustomers = dataSource.GetCustomerList().SelectSatisfying(isInLondon);

Of course we can also use delegates to achieve the same effect, by expanding the number of parameters. Of course this would be painful if we had to create a new delegate for each and every combination of parameters and return type. Fortunately C# 3.0 comes to the rescue by defining a range of generic delegates, provided we want less than three parameters, we just have to remember that the last type is the return type:

public delegate RT Func<RT>();
public delegate RT Func<T0, RT>(T0 a0);
public delegate RT Func<T0, T1, RT>(T0 a0, T1 a1);
public delegate RT Func<T0, T1, T2, RT>(T0 a0, T1 a1, T2 a2);
public delegate RT Func<T0, T1, T2, T3, RT>(T0 a0, T1 a1, T2 a2, T3 a3);

so we can use these to get lambdas to provide us with a specfication as before:

Func<Customer, string, bool> isCustomerIn = ( c, cty) => c.City == cty;
List<Customer> londonCustomers = dataSource.GetCustomerList().SelectSatisfying(isCustomerIn, "London");

again assuming something like:

public static List<Customer> SelectSatisfying(this List<Customer> existingCustomer s, Func<Customer, string, bool> specification, string city)
{
    List<Customer> customers = new List<Customer>();
    foreach (Customer customer in existingCustomers)
    {
       if (specification(customer, city))
          customers.Add(customer);
    }

    return customers;
}

although I find this becomes a little clumsier than the object version. Of course LINQ really provides the best way of doing list selection variants here:

public List<Customer> GetCustomersFor(string city)

{

    return
       from c in dataSource.GetCustomerList()
       where c.City == city
       select c;  

}

Composite Specifications

While encapsulating our specifications helps us to reduce duplication and improve maintainability, the real power of a specification comes from combining them. The common approach to combining specifications is to use the composite pattern. The composite pattern is a simple way of representing tree structures. Within a composite nodes are polymorphic, that is they share the same type (i.e. interface), but can have different implementations. Nodes may be leaf nodes or they may contain other nodes of the same type. The composite pattern lets us treat leaf nodes and compositions uniformly. What we need to know most, is that its great for lightwieght parsing of an expression.

Lets modify our specification interface to add some logical operations

public interface Specification<T>
{
    bool IsSatisfiedBy(T customer);
    Specification<T> And(Specification<T> other);
    //... more operators
}

public abstract class AbstractSpecification<T>: Specification<T>
{
    public Specification<T> And(Specification<T> other)
    {
        return new AndSpecification<T>(this, other);
    }

    abstract public bool IsSatisfiedBy(T customer);
}

public class AndSpecification<T> : AbstractSpecification<T>
{
    private Specification<T> rhs;
    private Specification<T> lhs;
    public AndSpecification(Specification<T> rhs, Specification<T> lhs)
    {
        this.rhs = rhs;
        this.lhs = lhs;
    }

    public override bool IsSatisfiedBy(T candidate)
    {
        return rhs.IsSatisfiedBy(candidate) &&
        lhs.IsSatisfiedBy(candidate);
    }
}

which lets us write code like this:

Specification<Customer> isInLondon = new CustomerCitySpecification("London").And(new CustomerCountrySpecification("UK"));
List<Customer> londonCustomers = dataSource.GetCustomerList().SelectSatisfying(isInLondon);
ObjectDumper.Write(londonCustomers, 1);

given something like this:

public static List<Customer> SelectSatisfying(this List<Customer> existingCustomers, Specification<Customer> specification)
{
    List<Customer> customers = new List<Customer>();
    foreach (Customer customer in existingCustomers)
    {
       if (specification.IsSatisfiedBy(customer))
          customers.Add(customer);
    }

    return customers;

}

 the line:

Specification<Customer> isInLondon = new CustomerCitySpecification("London").And(new CustomerCountrySpecification("UK"));

is expressive because it is readable and within the ubiquitous language of the domain.

Of course we could just write

Predicate<Customer> isCustomerInLondon = c => c.City == "London" && c.Country == "UK"

which is clearly as expressive, but is hard to re-use as it its parts are not composable, leading to duplication. So what we want is some way of making our predicates composable and re-usable without losing their expressiveness.

C# 3.0 lambda expressions can be represented either as delegates or as expressions, a parsed version of the lambda expression. So we can use expressions to store our lambda expressions and recombine them:

Expression<Predicate<Customer>> isCustomerInLondon = c => c.City == "London"
Expression<Predicate<Customer>> isInUK = c => c.Country == "UK"
Expression<Predicate<Customer>> isLondonUKCustomer = isCustomerInLondon.And(isInUK);

given something like:

public static Expression<Predicate<T>> And<T>(this Expression<Predicate<T>> rhs, Expression<Predicate<T>> lhs)
{
    //Parameters to both expressions need to be the same and of type T
    ParameterExpression parameter = Expression.Parameter(typeof(T), "p");
    var invokedExpression = Expression.Invoke (lhs, rhs.Parameters.Cast<Expression> ());
    return Expression.Lambda<Predicate<T>> (Expression.And (rhs.Body, invokedExpression), rhs.Parameters);
}

But once we start to write Expression<Predicate<Customer>> our abstractions are leaking into our fluent interface ruining the effect, so what we would like to do is wrap them up within a class when we create them to clean up the syntax. What we are looking for is something like:

ISpecification<Customer> isCustomerInLondon = new Specification<Customer>(c => c.City == "London");
ISpecification<Customer> isInUK = new Specification<Customer>(c => c.Country == "UK");

ISpecification<Customer> isLondonUKCustomer = isCustomerInLondon.And(isInUK);

List<Customer> londonCustomers = dataSource.GetCustomerList().SelectSatisfying(isLondonUKCustomer);

which we can do by writing the following

public interface ISpecification<T>
{
    Expression<Predicate<T>> Predicate {get;set;}
    ISpecification<T> And(ISpecification<T> other);
    bool IsSatisfiedBy(T customer);
}

abstract public class AbstractSpecification<T> : ISpecification<T>
{
    protected Expression<Predicate<T>> predicate;
    public abstract bool IsSatisfiedBy(T value);

    public Expression<Predicate<T>> Predicate
    {
        get
        {
            return predicate;
        }
        set
        {
           predicate = value;
        }
   }

   public ISpecification<T> And(ISpecification<T> other)
   {
        return new AndSpecification<T>(this, other);
   }

}

public class Specification<T> : AbstractSpecification<T>
{
    public Specification(Expression<Predicate<T>> predicate)
    {
        this.predicate = predicate;
    }

    public override bool IsSatisfiedBy(T value)
    {
       return predicate.Compile().Invoke(value);
    }
}

public class AndSpecification<T> : AbstractSpecification<T>
{
    private ISpecification<T> rhs;
    private ISpecification<T> lhs;
    public AndSpecification(ISpecification<T> rhs, ISpecification<T> lhs)
    {
       this.rhs = rhs;
       this.lhs = lhs;
    }

    public override bool IsSatisfiedBy(T candidate)
    {
        ParameterExpression parameter = Expression.Parameter(typeof(T), "p");
        var invokedExpression = Expression.Invoke (lhs.Predicate, rhs.Predicate.Parameters.Cast<Expression>());
        Expression<Predicate<T>> andPredicate = Expression.Lambda<Predicate<T>>(Expression.And (rhs.Predicate.Body, invokedExpression), rhs.Predicate.Parameters);
        return andPredicate.Compile().Invoke(candidate);
   }
}

The interesting part here is that use the expression tree that we have for both lambda expressions to merge the two expressions into one new one. You can find a post where Matt Warren talks about this on MSDN Forums. Metaprogramming can look a little scary, but we're just working at a slightly higher level than when writing the code directly. The key to this is that it makes specification composition 'simple'.

Of course IQueryable<T> is composable so that I can write something like this to achieve a similar effect for LINQ queries, by using a prior IQueryable. So given something like this:

IQueryable<Customer> customersInLondon = 

    from c in customers
         where c.City == city
         select c;

I can compose it by writing something like

IQueryable<Customer> customersInLondonUK =

     from c in customersInLondon
         where c.Country == "UK"
         select c;

Which means that given an IQueryable<Customer>, which might be an collection on a DB, or an IEnumerable<Customer> collection I called ToQueryable() on (see my previous post on Being Ignorant in LINQ to SQL as to why this is important) I can keep adding new specifications to the original query until I have my required test.

And of course you might simplify this as above by writing something like:

public static IQueryable<T> IsInCity(this IQueryable<T>  source, string country)
{
    return

      from c in source
         where c.City == city
         select c;

}

which enables something like:

IQueryable<Customer> customers = ... //db or collection source

IQueryable<Customer> londonCustomers = customers.IsInCity("London");

Specifications and Repositories

There is a link between specifications and repositories in that repositories often act as the collections against which specifications are applied to produce a set of matching results. Look over at Being Ignorant with LINQ to SQL if you need more on this. This tends mean we want code that looks something like this

DataContext context = new DataContext(ConfigurationManager.ConnectionStrings["NorthWind"].ConnectionString, Mapping.GetMapping());

UnitOfWork unitOfWork = new UnitOfWork(context);
CustomerRepository customerRepository = new CustomerRepository(unitOfWork);

ISpecification<Customer> customersInLondon = new Specification<Customer>(c => c.City == "London");
ISpecification<Customer> customersInUK = new Specification<Customer>(c => c.City == "UK");

ISpecification<Customer> customersInLondonUK = customersInLondon.And(customersInUK);

List<Customer> matchingCustomers = customerRepository.FindBySpecification(customersInLondonUK);

but I can't extend the repository by writing something like:

public List<Customer> FindBySpecification(ISpecification<Customer> specification)
{
    return
        (from c in Customers
        where specification.IsSatisfiedBy(c)
        select c).ToList();
}

because LINQ can't convert specification.IsSatisfiedBy(c) to valid SQL. But I can write:

public IEnumerable<Customer> FindBySpecification(ISpecification<Customer> specification)
{
    IQueryable<Customer> customerQuery = from c in Customers select c;
    IQueryable<Customer> restrictedCustomerQuery = customerQuery.Where<Customer>(specification.Predicate);
    return restrictedCustomerQuery.ToList();
}

The only slight wrinkle to get this to work is that I need to switch from using a Predicate<T> to a Func<T, bool> because IQueryable<T> does not expect the former, as follows:

public interface ISpecification<T>
{
    Expression<Func<T, bool>> Predicate {get;set;}
    ISpecification<T> And(ISpecification<T> other);
    bool IsSatisfiedBy(T customer);
    ISpecification<T> Not();
    ISpecification<T> Or(ISpecification<T> other);
}

abstract public class AbstractSpecification<T> : ISpecification<T>
{
    protected Expression<Func<T, bool>> predicate;
    public abstract bool IsSatisfiedBy(T value);

    public Expression<Func<T, bool>> Predicate
    {
        get
        {
            return predicate;
        }
        set
        {
            predicate = value;
        }
    }

    public ISpecification<T> And(ISpecification<T> other)
    {
        return new AndSpecification<T>(this, other);
    }

}
public class Specification<T> : AbstractSpecification<T>
{
    public Specification(Expression<Func<T, bool>> predicate)
    {
         this.predicate = predicate;
    }

    public override bool IsSatisfiedBy(T value)
    {
       return predicate.Compile().Invoke(value);
    }
}

public class AndSpecification<T> : AbstractSpecification<T>
{
    private ISpecification<T> rhs;
    private ISpecification<T> lhs;
    public AndSpecification(ISpecification<T> rhs, ISpecification<T> lhs)
    {
        this.rhs = rhs;
        this.lhs = lhs;

        ParameterExpression parameter = Expression.Parameter(typeof(T), "p");
        var invokedExpression = Expression.Invoke (lhs.Predicate, rhs.Predicate.Parameters.Cast<Expression>());
        Predicate = Expression.Lambda<Func<T, bool>>(Expression.And (rhs.Predicate.Body, invokedExpression), rhs.Predicate.Parameters);
    }

    public override bool IsSatisfiedBy(T candidate)
    {
        return Predicate.Compile().Invoke(candidate);
    }
}

and just for expressiveness we could use more C# 3.0 language features to improve the conciseness of our calling code:

var customersInLondon = new Specification<Customer>(c => c.City == "London");
var customersInUK = new Specification<Customer>(c => c.City == "UK");

var customersInLondonUK = customersInLondon.And(customersInUK);

or even

var customersInLondonUK = new Specification<Customer>(c => c.City == "London").And(new Specification<Customer>(c => c.City == "UK"));

Of course the win here is that our specification does not care if the datasource is in-memory or persisted elsewhere, it relies on the IQueryable<T> implementation to understand that, and just modifies that IQueryable.

Further Reading

Those of you with an interest in Dynamic Queries might want to check out Mike Taulty's post, and its follow ups here, and here. The major difference is that Mike uses string based expressions instead of lamdas to build his queries, modelling a different interaction style.

Matt Warren has a great series of posts over on his blog on buildling an IQueryable provider for those of you with an interest programming with expression trees, which pretty much explains the how of LINQ to SQL and as such is a great source on how to 'roll your own'.