Friday, February 11, 2011

Generic Producer Consumer Class in C#

Over the years, I implemented the Producer/Consumer pattern numerous times. Today, I had a need to implement it. Again. In the spirit of DRY, I decided to write it once and for all.

I began thinking about the requirements of a Producer/Consumer object. It had to be generic, so that it would be type safe. Users of the object needed to be able to specify the action that would be executed upon consumption. And I wanted an Enqueue method, which would be the entry point for producers. Here's my starting point:

public class ProducerConsumer<T>
{
public ProducerConsumer(Action<T> consumerAction) { /* snip */ }

public void Enqueue(T item) { /* snip */ }
}


Producers will use the Enqueue method. But I only want to allow one producer to enqueue at a time, so I'm going to need to introduce an object that I'll use to lock with. I'm also going to need a Queue to hold items.

public class ProducerConsumer<T>
{
private readonly Queue<T> queue = new Queue<T>();

private readonly object queueLocker = new object();

/* snip */

public void Enqueue(T item)
{
lock (this.queueLocker)
{
this.queue.Enqueue(item);
}
}
}


I'm going to have to think about how to consume the items. I'll want to do this on a separate thread, so I'll set this up in the constructor. The thread's method should be an infinite loop - it will dequeue items and hand them to the consumer action. I'll also need to ensure that only one thread is adding to or removing from the queue at a time - I'll do this by locking our queueLocker when I dequeue.

public class ProducerConsumer<T>
{
/* snip */

private readonly Action<T> consumerAction;

public ProducerConsumer(Action<T> consumerAction)
{
this.consumerAction = consumerAction;
new Thread(this.ConsumeItems) { IsBackground = true }.Start();
}

/* snip */

private void ConsumeItems()
{
while (true)
{
T nextItem;

lock (this.queueLocker)
{
nextItem = this.queue.Dequeue();
}

this.consumerAction(nextItem);
}
}
}


We have a problem here. What if there are no items in the queue? I'll need to check that before dequeuing. And if there are no items in the queue, I'll need to block until there are some items. So the Enqueue method will need to signal when an item is enqueued. I'll use an AutoResetEvent for this.

public class ProducerConsumer<T>
{
private readonly AutoResetEvent queueWaitHandle = new AutoResetEvent(false);

/* snip */

public void Enqueue(T item)
{
lock (this.queueLocker)
{
this.queue.Enqueue(item);

// After enqueuing the item, signal the consumer thread.
this.queueWaitHandle.Set();
}
}

private void ConsumeItems()
{
while (true)
{
T nextItem = default(T);

// Later on, we'll need to know whether there was an item in the queue.
bool doesItemExist;

lock (this.queueLocker)
{
doesItemExist = this.queue.Count > 0;
if (doesItemExist)
{
nextItem = this.queue.Dequeue();
}
}

if (doesItemExist)
{
// If there was an item in the queue, process it...
this.consumerAction(nextItem);
}
else
{
// ...otherwise, wait for the an item to be queued up.
this.queueWaitHandle.WaitOne();
}
}
}
}


That's everything! Now, to put it together:

public class ProducerConsumer<T>
{
private readonly Queue<T> queue = new Queue<T>();

private readonly object queueLocker = new object();

private readonly AutoResetEvent queueWaitHandle = new AutoResetEvent(false);

private readonly Action<T> consumerAction;

public ProducerConsumer(Action<T> consumerAction)
{
if (consumerAction == null)
{
throw new ArgumentNullException("consumerAction");
}

this.consumerAction = consumerAction;
new Thread(this.ConsumeItems) { IsBackground = true }.Start();
}

public void Enqueue(T item)
{
lock (this.queueLocker)
{
this.queue.Enqueue(item);

// After enqueuing the item, signal the consumer thread.
this.queueWaitHandle.Set();
}
}

private void ConsumeItems()
{
while (true)
{
T nextItem = default(T);

// Later on, we'll need to know whether there was an item in the queue.
bool doesItemExist;

lock (this.queueLocker)
{
doesItemExist = this.queue.Count > 0;
if (doesItemExist)
{
nextItem = this.queue.Dequeue();
}
}

if (doesItemExist)
{
// If there was an item in the queue, process it...
this.consumerAction(nextItem);
}
else
{
// ...otherwise, wait for the an item to be queued up.
this.queueWaitHandle.WaitOne();
}
}
}
}


To use it, instantiate a ProducerConsumer, passing in the action you want to be performed when an item is consumed. Then, just start enqueuing items.

void Main()
{
var producerConsumer = new ProducerConsumer<int>(i => Console.WriteLine(i));

Random random = new Random();

var t1 = new Thread(() =>
{
for (int i = 0; i < 100; i++)
{
producerConsumer.Enqueue(i);
Thread.Sleep(random.Next(0, 5));
}
});

var t2 = new Thread(() =>
{
for (int i = 0; i > -100; i--)
{
producerConsumer.Enqueue(i);
Thread.Sleep(random.Next(0, 5));
}
});

t1.Start();
t2.Start();

t1.Join();
t2.Join();

Thread.Sleep(50);
}


What I've got here is a very basic implementation. The download allows the user to start and stop the consumer thread, along with options concerning what to do when it is stopped. It also gives the user the ability to clear the queue.

Friday, January 14, 2011

Visual Studio Code Kata Project Template

I just attended CodeMash, and I was really impressed by the pre-compiler all-day session "Software Craftsmanship" by Steve Smith (@ardalis) and Brendan Enrick (@brendoneus). During the session, we did several coding katas. The process for each kata was the same: start by writing a single test, then, as simply as possible, make the test pass. For example, don't do any logic, just return the expected result for the test. Then write another test, and, again, as simply as possible, make it (and the other one) pass. Continue until all use cases have corresponding tests, and all tests pass. This was probably my favorite session at CodeMash: I've known what TDD was in an abstract way, but until now, I didn't "get" it. Kudos to Steve and Branden.

However, there was one thing that slightly annoyed me during this session: when we started a new kata, I first had to create a new project/solution in Visual Studio (a class library), then delete Class1.cs, then add a reference to NUnit, then add a new class (the one that performs the kata), then add another class (the test class), and finally, decorate the test class with [TestFixture]. After all of this, I was ready to start.

I am far too lazy to go through all of these steps for each kata. So I created a Visual Studio project template to handle these routine, boring steps for me.

This was my first attempt at creating a VS project templat, so there was some fumbling around. I googled around and found these two MSDN articles: How to: Create Project Templates and Template Parameters. These two articles got me most of the way there.

After creating a tedious kata project as described above, I used Visual Studio's Export Template (File > Export Template...). After extracting the resulting .zip file, I hand modified the two .cs files, the .vstemplate, and the .csproj file so that the name of the project would be used for the kata and kata test classes. I then re-zipped the files and copied the .zip to the Visual Studio templates directory (located at C:\Users\<UserName>\Documents\Visual Studio 2010\Templates\ProjectTemplates for VS 2010 on Windows 7).

That's it! Now, when I get the urge to do a code kata, I can get started in seconds instead of minutes.

Kata Template.zip

EDIT: I've made the template version-agnostic for NUnit (it had been looking for a specific version of NUnit). This change should make usable for any version of NUnit. However, this will probably cause problems for those who have multiple versions of NUnit installed. In that case, unzip the template and edit KataProject.csproj. Change the NUnit reference to point to the version that you need. Rezip the files and put it in the ProjectTemplates directory and you should be good to go.