Tuesday, October 15, 2013

Generic Producer Consumer Class - Now on NuGet

I had the need again recently to use the producer-consumer pattern, so I got to work. I found the post I made a couple years ago, downloaded the code from it, and copied/pasted the code into my project (the only thing changing being the namespace). Not a big deal, but it takes a few minutes. I've done this process in at least two other projects, and it's annoying. I also really don't like that I've made exact duplicates of this class in multiple places. What's worse, I've made little improvements to it in each of the projects, so the codebases are beginning to diverge. If only there was a way to package it up and make it available, for free, to anyone that wanted it. And if only there was a way to maintain a historical record the different "versions" of the class.

If only.

So anyway, I created a NuGet package for the producer/consumer class. It's called RandomSkunk.ProducerConsumer. And the project is now on GitHub.

I shoulda done this a long time ago.

Saturday, May 25, 2013

Circular references between class libraries

Note: the demo project for this blog entry is on GitHub: https://github.com/bfriesen/CircularReferenceProblem.

Recently at work, we ran into a situation where we were building an internal library. Most of the library was written in C#, but some of it - complex statistical calculations - was better suited for F#. This posed a problem because the F# code needed to use some of the objects that were defined in the C# code, and the C# code needed to call the F# calculation code. Because C# and F# cannot exist in the same project, we had two projects on our hands, each of which needed to reference the other. This would cause a circular reference, and Visual Studio won't allow this to happen.

This post describes a similar situaion, and a pretty elegant solution to the problem.

The scenario

In our fake scenario, we're building a library with C# that provides a Calculator class. This class has one method, Add, that takes two Number objects. It should convert those Number objects into double values and return the result of adding those values. Now, you're going to have to suspend disbelief for a minute, but, for some reason, the developers had great difficulty in converting a Number object into a double. They determined that the best way of doing the converting was to use F#. They have created a NumberConverter class to handle this parsing.

The Number class (in the C# project):

public class Number
{
    public string Value { get; set; }
}

The NumberConverter class (in the F# project):

public class NumberConverter
{
    // Pretend really hard that this is written in F#. :)
    public double Parse(Number number)
    {
        return double.Parse(number.Value);
    }
}

The Calculator class (in the C# project):

public class Calculator
{
    private readonly NumberConverter _numberConverter;

    public Calculator()
    {
        _numberConverter = new NumberConverter();
    }

    public double Add(Number lhs, Number rhs)
    {
        return _numberConverter.Parse(lhs)
            + _numberConverter.Parse(rhs);
    }
}

The problem

.NET does not allow us to put F# code in a C# project. So we'll need to create a separate F# project. But this add a new problem: the Calculator will need to call the F# Add method of the NumberConverter class. However, the NumberConverter uses the Number class, which is defined in the C# project. This is at the heart of the circular reference problem.

Ordinarily (i.e. if this was an application, not a library), I would solve this problem by using a standard IoC container, such as Ninject. In the composition root of the application, we would wire up our dependencies (INumberConverter to NumberConverter). Then, when we needed something with dependencies, we'd ask the IoC container for it, and it would handle injecting the dependencies. But this won't work here, because we have a library, not an application. In a library, there is no composition root. So, we're stuck. Is there a way around this conumdrum? As luck would have it, there is.

The solution

To start, we'll create an interface, INumberConverter, in the C# project. Then we'll have the F# project reference the C# project. And we'll the NumberConverter class so that it inherits from INumberConverter. And we'll need to change our Calculator class too. We'll use Dependency Injection to inject an instance of INumberConverter into the Calculator.

The INumberConverter interface (in the C# project):

public interface INumberConverter
{
    double Parse(Number number);
}

The new NumberConverter class (in the F# project):

public class NumberConverter : INumberConverter
{
    // Pretend really hard that this is written in F#. :)
    public double Parse(Number number)
    {
        return double.Parse(number.Value);
    }
}

The Calculator class (in the C# project):

public class Calculator
{
    private readonly INumberConverter _numberConverter;

    public Calculator(INumberConverter numberConverter)
    {
        _numberConverter = numberConverter;
    }

    public double Add(Number lhs, Number rhs)
    {
        return _numberConverter.Parse(lhs)
            + _numberConverter.Parse(rhs);
    }
}

A new problem

This totally works, but we've eliminated the default constructor in Calculator. And that sucks. We really want that default constructor because we're making a reusable library and it is unreasonable to expect our library's consumers to have pass an inheritor of INumberConverter in to Calculator.

A new solution

We need to find a way to get that default constructor back in. What if Calculator had some magic way to get ahold of a NumberConverter? What would we do if we had that magic? We would probably have that default constructor pass the NumberConverter to the non-default constructor. Something like this:

public class Calculator
{
    private readonly INumberConverter _numberConverter;

    public Calculator(INumberConverter numberConverter)
    {
        _numberConverter = numberConverter;
    }

    public Calculator()
        : this(Magic.GetNumberConverter()) // <-- Magic!
    {
    }

    public double Add(Number lhs, Number rhs)
    {
        return _numberConverter.Parse(lhs)
            + _numberConverter.Parse(rhs);
    }
}

What if we made the GetNumberConverter method generic? Then we'd end up with Magic.Get<INumberConverter>(). And what if we renamed Magic to ServiceLocator? Well, then we'd have ServiceLocator.Get<INumberConvert>(), and we would suddenly be using the Service Locator pattern. Now, I'm not a huge fan of the Service Locator pattern. In fact, many people call it an anti-pattern. But in this case, I think it works. Especially if you declare the ServiceLocator as internal. Here's the general shape of the ServiceLocator class:

internal static class ServiceLocator
{
    public static T Get<T>()
    {
        return ??????????;
    }
}

The final piece of the puzzle

To solve the final piece of the puzzle, we're going to use Managed Extensibility Framework, or MEF. The reason why I chose MEF for this was because it's part of .NET's base class library (BCL). Because we're building a library, we don't want to force its consumers to use some third-party library. In other words, MEF is always available (at least to .NET 4.0 applications). The first thing we'll do is decorate the NumberConverter class with MEF's ExportAttribute, passing in typeof(INumberConverter. This tells MEF that we want an instance of NumberConverter to be returned when we ask MEF for a INumberConverter.

[Export(typeof(INumberConverter))] // <-- Add ExportAttribute
public class NumberConverter : INumberConverter
{
    // Pretend really hard that this is written in F#. :)
    public double Parse(Number number)
    {
        return double.Parse(number.Value);
    }
}

The final step is to tell the ServiceLocator to use MEF in order to be able to return an instance of INumberConverter. This is where MEF really shines. We can instruct MEF to scan the directory of the currently executing code for DLLs, and when it finds any, to scan the DLLs for types decorated with ExportAttribute. Without going into the details of how MEF does this, here is the final version of ServiceLocator:

internal static class ServiceLocator
{
    private static readonly CompositionContainer _container =
        new CompositionContainer(new DirectoryCatalog(@".\"));

    public static T Get<T>()
    {
        return _container.GetExportedValue<T>();
    }
}

Putting it all together

Let's create a simple console application to use the Calculator. We'll add references to both the C# project and the F# project. Then, all we need to do is instantiate a Calculator and pass in two Number objects.

static void Main()
{
    var firstNumber = new Number { Value = "123.456" };
    var secondNumber = new Number { Value = "654.321" };

    var calculator = new Calculator();
    var result = calculator.Add(firstNumber, secondNumber);

    Console.WriteLine(result);
    Console.ReadKey();
}

VoilĂ ! We have resolved the circular dependency problem, and the consumers of our library are none the wiser.

Tuesday, January 29, 2013

Flattening Polymorphism with AutoMapper

I've been working on a project that is exposed via a service layer. Trying to "do the right thing", I've created a DTO project that contains everything that is sent over the wire from the service layer (which happens to be ServiceStack). I really detest writing mapping code, so I'm using AutoMapper to take care of that for me.

I ran into an issue though - some of my entities are polymorphic. That is, I have an abstract base class and two inheritors of that base class. And I didn't want my DTOs to be polymorphic - I wanted them flat. I solved the problem by creating two methods and an interface.

Both methods require that the DTO inherit from this interface.
public interface IPolymorphicDto
{
    string Type { get; set; }
}

Here's the first method, which takes polymorphic objects and maps them to a flat DTO.

public static void CreatePolymorphicToFlatMap<
    TSourceBase,
    TSourceInheritorA,
    TSourceInheritorB,
    TDestination>()
    where TSourceInheritorA : TSourceBase
    where TSourceInheritorB : TSourceBase
{
    // Map the abstract source type to the destination DTO
    // and include the two inheritor source types.
    var mappingExpression =
        Mapper.CreateMap<TSourceBase, TDestination>()
            .Include<TSourceInheritorA, TDestination>()
            .Include<TSourceInheritorB, TDestination>();

    // Tell AutoMapper to ignore each property type that exists
    // in either inheritor but not the abstract base. (we'll
    // take care of them later)
    foreach (var name in
        typeof(TSourceInheritorA).GetProperties()
            .Where(p => p.DeclaringType
                == typeof(TSourceInheritorA))
            .Select(p => p.Name)
        .Union(typeof(TSourceInheritorB).GetProperties()
            .Where(p => p.DeclaringType
                == typeof(TSourceInheritorB))
            .Select(p => p.Name)))
    {
        mappingExpression.ForMember(name, options =>
            options.Ignore());
    }

    // Tell AutoMapper how to map the inheritor's properties.
    Mapper.CreateMap<TSourceInheritorA, TDestination>();
    Mapper.CreateMap<TSourceInheritorB, TDestination>();
}
The second method takes a flattened DTO and maps it into a polymorphic objects.
public static void CreateFlatToPolymorphicMap<
    TSource,
    TDestinationBase,
    TDestinationInheritorA,
    TDestinationInheritorB>()
    where TSource : IPolymorphicDto
    where TDestinationBase : class
    where TDestinationInheritorA : TDestinationBase, new()
    where TDestinationInheritorB : TDestinationBase, new()
{
    Mapper.CreateMap<TSource, TDestinationBase>()
        // We can't rely on AutoMapper to create an instance
        // of TDestinationBase, since it's abstract. So we
        // need to tell it how to do so.
        .ConstructUsing(source =>
        {
            // We'll use a bit of reflection to create the
            // instance.
            return (TDestinationBase)
                Activator.CreateInstance(
                    Type.GetType(source.Type));
        })
        // We also can't rely on AutoMapper to figure out
        // how to map the properties of the inheritors.
        // We're creating a TDestinationBase map here, so
        // AutoMapper won't know how to map the properties
        // that exist only in the inheritors. We'll solve
        // the problem by checking the Type property of the
        // source object (note that we put a generic constraint
        // on TSource - it must be a IPolymorphicDto). If the
        // Type property of TSource matches the full name of
        // TDestinationInheritorA, then return whatever
        // AutoMapper maps to when we tell it we're mapping to
        // TDestinationInheritorA. Otherwise, have Automapper
        // map to TDestinationInheritorB.
        .ConvertUsing(source =>
            source.Type
                == typeof(TDestinationInheritorA).FullName
            ? (TDestinationBase)
                Mapper.Map<TDestinationInheritorA>(source)
            : (TDestinationBase)
                Mapper.Map<TDestinationInheritorB>(source));

    // Finally, we need to tell AutoMapper how to map to the
    // inheritors.
    Mapper.CreateMap<TSource, TDestinationInheritorA>();
    Mapper.CreateMap<TSource, TDestinationInheritorB>();
}
So now we have our methods. Let's define some classes to map.
//Domain Objects
public class Customer
{
    private readonly List<Order> _orders = new List<Order>();

    public List<Order> Orders
    {
        get { return _orders; }
    }
}

public abstract class Order
{
    public int Id { get; set; }
}

public class OnlineOrder : Order 
{
    public string Referrer { get; set; }
}

public class MailOrder : Order
{
    public string City { get; set; }
}

//Dtos
public class CustomerDto
{
    private readonly List<OrderDto> _orders
        = new List<OrderDto>();

    public List<OrderDto> Orders
    {
        get { return _orders; }
    }
}

public class OrderDto : IPolymorphicDto
{
    public int Id { get; set; }
    public string Type { get; set; }
    public string Referrer { get; set; }
    public string City { get; set; }
}
And now we can set up our mappings.
Mapper.CreateMap<Customer, CustomerDto>();
CreatePolymorphicToFlatMap<
    Order, OnlineOrder, MailOrder, OrderDto>();
Mapper.CreateMap<CustomerDto, Customer>();
CreateFlatToPolymorphicMap<
    OrderDto, Order, OnlineOrder, MailOrder>();

// Ask AutoMapper to validate our mapping. If something is
// wrong, this will throw an exception.
Mapper.AssertConfigurationIsValid();
Finally, we can map back and forth with ease.
Customer customer = new Customer();
Order onlineOrder = new OnlineOrder
{
    Id = 1,
    Referrer = "google"
};
Order mailOrder = new MailOrder
{
    Id = 2,
    City = "Detroit"
};
customer.Orders.Add(onlineOrder);
customer.Orders.Add(mailOrder);
// customer will have two Orders: one of type OnlineOrder, with
// an Id of 1 and a Referrer of "google"; and one of type
// MailOrder, with an Id of 2 and a City of "Detroit";

var mapped = Mapper.Map<CustomerDto>(customer);    
// mapped will have two Orders: one with an Id of 1, a Type of
// OnlineOrder, a Referrer of "google", and a City with a null
// value; and one with an Id of 2, a Type of MailOrder, a
// Referrer with a null value, and a City of "Detroit".

var mappedBack = Mapper.Map<Customer>(mapped);
// mappedBack will be identical to the original customer.

Thursday, August 23, 2012

The Nomadic Monad or: How I Learned to Stop Worrying and Love the Burrito (Part 3)

This is the third in a series of posts about monads (part 1, part 2). I happen to be a .NET developer, so I use C# in my examples, but the concept applies to any language that has first-class functions. Code for the series is at github.

In the last post, we covered over half of the items in Wikipedia's formal definition of a monad:

  • Formally, a monad consists of a type constructor M and two operations, bind and return.
  • The operations must fulfill several properties to allow the correct composition of monadic functions (i.e. functions that use values from the monad as their arguments or return value).
  • In most contexts, a value of type M a can be thought of as an action that returns a value of type a.
  • The return operation takes a value from a plain type a and puts it into a monadic container of type M a.
  • The bind operation chains a monadic value of type M a with a function of type a ? M b to create a monadic value of type M b, effectively creating an action that chooses the next action based on the results of previous actions.

Let's break down the requirement for the Bind function. It

...chains a monadic value of type M a...
That says to me that we need to have an instance method in our monad. Or, we could use an extension method to achieve the same result. I prefer to use an extension method, so that's what we'll do. Let's create an extension method:

public static ??? Bind<T>(this Monad<T> monad, ???)

Not a bad start. Let's skip to the return type. We'll need

...to create a monadic value of type M b...
That doesn't look so hard. We've already used our Monad<T> class in place of M a. But, since we already used T we won't be able to use it again. But, since it's a good idea to use standard C# naming conventions when we're writing C# code (when in Rome...), our generic type should at least start with the letter 'T'. How about TResult, since it is the result of our Bind method?

public static Monad<TResult> Bind<T, TResult>(
    this Monad<T> monad,
    ???)

But what of the second parameter? It looks like it needs we need to be connecting

...with a function of type a ? M b...
Why, if I didn't know better, I'd say that looks an awful like a lambda expression (spoiler alert: it is). And we already know what a and M b mean to us: T and Monad<TResult>. It would be great if we were able to pass in a lambda expression that takes an T and returns a Monad<TResult>. But what is a lambda expression? Just a convenient way of creating a function. Hmmmm, function. I know - why don't we take a Func<T, Monad<TResult>> as our parameter type! We just put the function in functional. So what does our extension method look like now?

public static Monad<TResult> Bind<T, TResult>(
    this Monad<T> monad,
    Func<T, Monad<TResult>> resultSelector)

Finally, we'll need to fill in the body. Our requirements state that we need to take our Monad<T> and transform it into a Monad<TResult> by using a Func<T, TResult>. That function needs a <T> argument passed to it. Haven't we seen an instance of <T> before? Of course - in the Value property of our Monad<T> class. With that, I think we have got enough information to fill out our Bind method:

public static Monad<TResult> Bind<T, TResult>(
    this Monad<T> monad,
    Func<T, Monad<TResult>> resultSelector)
{
    return resultSelector(monad.Value);
}

I think we're really getting somewhere now! How would we use our spiffy new Bind function? I'd think that we would use it something like this:

Monad<int> m1 = 128.ToMonad();
Monad<Type> m2 = m1.Bind(x => x.GetType().ToMonad());
Console.WriteLine(
    "The type of the original monad is: {0}.",
    m2.Value);

We've taken a Monad of type int, and bound it to a Monad of type Type by calling Bind and passing in a lambda expression that takes a T and returns a Monad<Type>. There's one thing I don't like about this usage though: the call to ToMonad(). I'd prefer it if our Bind function had, as its parameter, a Func<T, TResult>. But I also want to meet the requirement for being a monad. Why don't we create an overload of Bind to do just that? I'd like our existing Bind method to remain the source of truth, so we'll have our new one call it.

public static Monad<TResult> Bind<T, TResult>(
    this Monad<T> monad,
    Func<T, TResult> resultSelector)
{
    return monad.Bind(m => resultSelector(m).ToMonad());
}

This is getting a little funky. We're doing a few things here: 1) we're passing a new lambda expression to the other Bind method; 2) in the body of the new lambda expression, we're executing the function that was passed in; and 3) we're turning the result of into a Monad<TResult> by calling our extension method, ToMonad. We're not actually executing the passed-in function, we're passing it into another function that we pass to the other Bind method. Clear as mud, right? So why go through all this trouble? To me, it expresses intent better, and I think this makes it worth it. We can use it like this:

Monad<int> m1 = 128.ToMonad();
Monad<Type> m2 = m1.Bind(x => x.GetType());

Console.WriteLine(
    "The type of the original monad is: {0}.",
    m2.Value);

This is all well and good, but it seems kind of tedious to have to create a new variable each time we call Bind. Didn't the requirement mention something about chaining? As in method chaining? Why don't we try that?

var m =
    128.ToMonad()
    .Bind(x => x.GetType())
    .Bind(x => new string(x.ToString().Reverse().ToArray()));

Console.WriteLine(
    "The backwards string representation of "
    + "the type of the original monad is: {0}.",
    m.Value);

We're taking the number 128 wrapping it in Monad<int>, binding it to a Monad<Type> with a function that takes any value and returns its type, and binding that to a Monad<string> with a function that takes any value and returns its string representation backwards. Pretty cool, huh?

We're headed for the home stretch now. It's time to address the final bullet point:

The operations must fulfill several properties to allow the correct composition of monadic functions (i.e. functions that use values from the monad as their arguments or return value).
It seems to me like we're already doing this. We're composing a monad from a series of monadic functions. But perhaps we can make our intent more clear - calling Bind over and over again doesn't exactly look pretty. What if we extracted the calls to Bind into methods that expressed their intent more clearly? How about using some extensions methods? Something like this?

public static Monad<Type> ToType<T>(this Monad<T> monad)
{
    return monad.Bind(m => m.GetType());
}

public static Monad<string> ToReversedString<T>(
    this Monad<T> monad)
{
    return monad.Bind(m =>
        new string(m.ToString().Reverse().ToArray()));
}

Now we can replace our Bind statements:

var m =
    128.ToMonad()
    .ToType()
    .ToReversedString();

Console.WriteLine(
    "The backwards string representation of "
    + "the type of the original monad is: {0}.",
    m.Value);

That about wraps it up for today. Next time, we'll discuss the three monadic laws.(Hopefully we follow them!)

Sunday, July 1, 2012

The Nomadic Monad or: How I Learned to Stop Worrying and Love the Burrito (Part 2)

This is the second in a series of posts about monads (part 1, part 3). I happen to be a .NET developer, so I use C# in my examples, but the concept should apply to any language that has first-class functions. Code for the series is at github.

We left off with the definition of a monad. Let's review:

  • Formally, a monad consists of a type constructor M and two operations, bind and return.
  • The operations must fulfill several properties to allow the correct composition of monadic functions (i.e. functions that use values from the monad as their arguments or return value).
  • In most contexts, a value of type M a can be thought of as an action that returns a value of type a.
  • The return operation takes a value from a plain type a and puts it into a monadic container of type M a.
  • The bind operation chains a monadic value of type M a with a function of type a → M b to create a monadic value of type M b, effectively creating an action that chooses the next action based on the results of previous actions.

Let's begin by addressing the first bullet point.

Formally, a monad consists of a type constructor M and two operations, bind and return.
As I understand it, a type constructor in .NET is analogous to a generic type. And with that little bit of information, we can start writing code. Let's start our project by creating a generic type, Monad<T>.

public class Monad<T>
{
    
}
github commit

With that, we've satisfied the first half of the first bullet point. We still need to define bind and return, but, since the fourth and fifth bullet points expand on each function, let's declare the first bullet point complete (that was easy!).

Let's move on to the third bullet point (I know I'm skipping around here):

In most contexts, a value of type M a can be thought of as an action that returns a value of type a.
I may be taking some liberties with the formal definition, but this screams "PROPERTY!" to me. So let's add a read-only property, Value, of type T, backed by a readonly field of the same type. And let's provide a single constructor whose parameter is of type T. We want to force people to do the right thing - always provide the monad with a value, and never (ever!) change the its value. Our monad follows the spirit of functional programming - it is immutable.

public class Monad<T>
{
    private readonly T _value;

    public Monad(T value)
    {
        _value = value;
    }

    public T Value
    {
        get { return _value; }
    }
}
github commit

As you might expect, usage of our class is very simple, as it does nearly nothing.

var m = new Monad<int>(128);
Console.WriteLine("The value of m is {0}.", m.Value);
github commit

As boring as the code is - the declaration of class Monad<T> and its usage - we have actually done something quite interesting here. If you think about it, just by having a generic type and a constructor, it fulfills the fourth bullet point:

The return operation takes a value from a plain type a and puts it into a monadic container of type M a.
We took a value, 128, and wrapped it in our class, Monad<int>. Again, we've taken some liberties with the formal definition. Instead of a actual return function, we have a constructor. I don't think it's too much of a stretch.

As little as our class does, I'm not exactly happy with it. More specifically, I'm not happy with its usage. If we already have an value that we want to wrap in our monad, why should we have to explicitly provide the generic type argument in the constructor? Why can't C# just figure it out? In short, that's just how constructors work in C#. And you'll just have to get used to it. But we can do better. We can provide a nice, easy-to-use workaround: an extension method.

public static Monad<T> ToMonad<T>(this T value)
{
    return new Monad<T>(value);
}
github commit

It makes creating an instance of our class much easier on the eyes:

var m = 128.ToMonad();
Console.WriteLine("The value of m is {0}.", m.Value);
github commit

That about wraps it up for this post. We have satisfied over half (2 2/3 by my count) of the five conditions for a monad. We have a generic class. It can wrap any value of any type. And it exposes its value as a read-only property. But we don't have a monad yet. So what's left? The bind method, and the ability to chain monadic functions. Stay tuned, things are about to get interesting.

Friday, June 29, 2012

The Nomadic Monad or: How I Learned to Stop Worrying and Love the Burrito (Part 1)

This is the first in a series of posts about monads (part 2, part 3). I happen to be a .NET developer, so I'll use C# in my examples, but the concept should apply to any language that has first-class functions.

So why should I care about monads? Aren't they only found in those weird academic languages (like Haskell) that nobody actually uses in production code? Actually, no. If you're a .NET developer like me, you've already been using monads. For almost 5 years.

I first became aware of the existence of the concept of monads after attending Kevin Hazard's CodeMash 2011 talk, What the Math Geeks Don't Want You to Know about F#. Somewhere amid the concepts of pattern matching, closures, and discriminated unions, he mentioned monads. What he said was something the line of, "...then there are monads - I'm not going to get into them because this is an intro-level talk." This caught my attention, and, since I had never heard of monads, I wrote a single word - monad - in my notebook. And forgot about it as soon as I got home. Several months later, somebody mentioned monads in a tweet. I thought to myself, "Hmmm, haven't I heard that word before?" Then I forgot about it again. Several months after that, I came across it again on Twitter. This time, I got fed up with myself - there was a concept out there that I was completely ignorant of. It was time to educate myself. Thus, my quest began.

So what is a monad? And what do they have to do with burritos? First, and least importantly, burritos. If you spend any non-trivial amount of time googling monads, you will undoubtedly come across someone comparing them to burritos. Go ahead, google "monad" "burrito" - I get 847,000 results. Interesting, but not really relevant to what a monad is. For that, let's go the source of all knowledge, Wikipedia. I've taken the liberty of bulletizing its definition of a monad:

  • Formally, a monad consists of a type constructor M and two operations, bind and return.
  • The operations must fulfill several properties to allow the correct composition of monadic functions (i.e. functions that use values from the monad as their arguments or return value).
  • In most contexts, a value of type M a can be thought of as an action that returns a value of type a.
  • The return operation takes a value from a plain type a and puts it into a monadic container of type M a.
  • The bind operation chains a monadic value of type M a with a function of type a → M b to create a monadic value of type M b, effectively creating an action that chooses the next action based on the results of previous actions.

Wait, what?! This doesn't exactly make sense at first. But have no fear, it will all come together. Beginning with the next post, I'll begin dissecting each bullet point. Trust me - it'll all make sense by the end.

Friday, February 11, 2011

Generic Producer Consumer Class in C#

Over the years, I implemented the Producer/Consumer pattern numerous times. Today, I had a need to implement it. Again. In the spirit of DRY, I decided to write it once and for all.

I began thinking about the requirements of a Producer/Consumer object. It had to be generic, so that it would be type safe. Users of the object needed to be able to specify the action that would be executed upon consumption. And I wanted an Enqueue method, which would be the entry point for producers. Here's my starting point:

public class ProducerConsumer<T>
{
public ProducerConsumer(Action<T> consumerAction) { /* snip */ }

public void Enqueue(T item) { /* snip */ }
}


Producers will use the Enqueue method. But I only want to allow one producer to enqueue at a time, so I'm going to need to introduce an object that I'll use to lock with. I'm also going to need a Queue to hold items.

public class ProducerConsumer<T>
{
private readonly Queue<T> queue = new Queue<T>();

private readonly object queueLocker = new object();

/* snip */

public void Enqueue(T item)
{
lock (this.queueLocker)
{
this.queue.Enqueue(item);
}
}
}


I'm going to have to think about how to consume the items. I'll want to do this on a separate thread, so I'll set this up in the constructor. The thread's method should be an infinite loop - it will dequeue items and hand them to the consumer action. I'll also need to ensure that only one thread is adding to or removing from the queue at a time - I'll do this by locking our queueLocker when I dequeue.

public class ProducerConsumer<T>
{
/* snip */

private readonly Action<T> consumerAction;

public ProducerConsumer(Action<T> consumerAction)
{
this.consumerAction = consumerAction;
new Thread(this.ConsumeItems) { IsBackground = true }.Start();
}

/* snip */

private void ConsumeItems()
{
while (true)
{
T nextItem;

lock (this.queueLocker)
{
nextItem = this.queue.Dequeue();
}

this.consumerAction(nextItem);
}
}
}


We have a problem here. What if there are no items in the queue? I'll need to check that before dequeuing. And if there are no items in the queue, I'll need to block until there are some items. So the Enqueue method will need to signal when an item is enqueued. I'll use an AutoResetEvent for this.

public class ProducerConsumer<T>
{
private readonly AutoResetEvent queueWaitHandle = new AutoResetEvent(false);

/* snip */

public void Enqueue(T item)
{
lock (this.queueLocker)
{
this.queue.Enqueue(item);

// After enqueuing the item, signal the consumer thread.
this.queueWaitHandle.Set();
}
}

private void ConsumeItems()
{
while (true)
{
T nextItem = default(T);

// Later on, we'll need to know whether there was an item in the queue.
bool doesItemExist;

lock (this.queueLocker)
{
doesItemExist = this.queue.Count > 0;
if (doesItemExist)
{
nextItem = this.queue.Dequeue();
}
}

if (doesItemExist)
{
// If there was an item in the queue, process it...
this.consumerAction(nextItem);
}
else
{
// ...otherwise, wait for the an item to be queued up.
this.queueWaitHandle.WaitOne();
}
}
}
}


That's everything! Now, to put it together:

public class ProducerConsumer<T>
{
private readonly Queue<T> queue = new Queue<T>();

private readonly object queueLocker = new object();

private readonly AutoResetEvent queueWaitHandle = new AutoResetEvent(false);

private readonly Action<T> consumerAction;

public ProducerConsumer(Action<T> consumerAction)
{
if (consumerAction == null)
{
throw new ArgumentNullException("consumerAction");
}

this.consumerAction = consumerAction;
new Thread(this.ConsumeItems) { IsBackground = true }.Start();
}

public void Enqueue(T item)
{
lock (this.queueLocker)
{
this.queue.Enqueue(item);

// After enqueuing the item, signal the consumer thread.
this.queueWaitHandle.Set();
}
}

private void ConsumeItems()
{
while (true)
{
T nextItem = default(T);

// Later on, we'll need to know whether there was an item in the queue.
bool doesItemExist;

lock (this.queueLocker)
{
doesItemExist = this.queue.Count > 0;
if (doesItemExist)
{
nextItem = this.queue.Dequeue();
}
}

if (doesItemExist)
{
// If there was an item in the queue, process it...
this.consumerAction(nextItem);
}
else
{
// ...otherwise, wait for the an item to be queued up.
this.queueWaitHandle.WaitOne();
}
}
}
}


To use it, instantiate a ProducerConsumer, passing in the action you want to be performed when an item is consumed. Then, just start enqueuing items.

void Main()
{
var producerConsumer = new ProducerConsumer<int>(i => Console.WriteLine(i));

Random random = new Random();

var t1 = new Thread(() =>
{
for (int i = 0; i < 100; i++)
{
producerConsumer.Enqueue(i);
Thread.Sleep(random.Next(0, 5));
}
});

var t2 = new Thread(() =>
{
for (int i = 0; i > -100; i--)
{
producerConsumer.Enqueue(i);
Thread.Sleep(random.Next(0, 5));
}
});

t1.Start();
t2.Start();

t1.Join();
t2.Join();

Thread.Sleep(50);
}


What I've got here is a very basic implementation. The download allows the user to start and stop the consumer thread, along with options concerning what to do when it is stopped. It also gives the user the ability to clear the queue.