Thread safe Queue with no allocations

This post might be pretty specialized stuff, but I thought I'd write it up and post here just in case anyone else finds it useful.

I send a lot of messages over the network and the messages were taking too long to serialize. A solution to this problem is to move the serialization to another thread so it doesn't block my Update/FixedUpdate.

I need to send the message from my game object to the network send thread, so I used a Queue to do this. This needs to be thread safe so at first I tried using a lock for the .Net Queue. Unfortunately, due to contention between the threads I found it blocking all the time with really harms the framerate.

Instead or Queue, I tried the ConcurrentQueue from Mono. This was pretty easy to port, I just had to get rid of an interface or two. But this had the problem or allocating all the time. Allocations can be pretty bad in Unity causing delays if you're doing them every frame. Again, this can worsen your framerate.

In the end, I went for something called the Disruptor. It's a lock free data structure that can be used as a queue that doesnt do any allocations. It's from low-latency finance tech but works great for my use case.

In case anyone else might find it useful, I've posted my code here: https://github.com/dave-hillier/disruptor-unity3d with more details about the motivation behind it. I've also done a simple benchmark which shows the performance improvements it gives.

 

'll check latter. Thanks for sharing anyway ;]

 

Bookmarking in case I need to use this.

Just starting to explore threading possibilities for my infinite terrain + biome generator. Currently I create a new thread every time I do a task. But I have a suspicion this may turn around and bite me. Really got to read up on what happens to abandoned threads.

 

How hard would it be to configure this to work with multiple consumers?

 

Thanks! I had written my own but was really basic and needed to expand it, this is perfect it will save me lots of work!

 

What are you considering abandoned threads?

If a thread has code in it, and the code completes, then the thread is finished and removed.

BUT, if you have a thread that doesn't end. Maybe it loops on some interval, or it has some wait condition that could place it in wait indefinitely... then the thread will just stay in memory sitting around. Worse, if the thread isn't marked as a 'background' thread, it'll remain in existence even AFTER you close your program, since non-background threads can persist after you close the program, it'll show up as its own process in the task manager.

 

Very interesting. Something that should be noted in the readme is that this implementation is safe as long as the producer runs in a single thread and the consumer runs in a single thread. eg if dequeue() or enqueue is called from multiple threads each, the result will be unpredictable. The producer and consumer can be in 2 different threads of course.

Also, using `Thread.SpinWait(1);` burns CPU cycles. For example, if the consumer is blocked inside `dequeue()` waiting for a new entry, the CPU usage is really high. This will not show in Unity's profiler, if the consumer is running on a thread other than the main thread. You can test this scenario if you start the producer with a delay of a few seconds.

I have created an improved version of the queue, that replaces `SpinWait` with `AutoResetEvent`. It tries to signal the event only when it is needed to avoid wasting CPU, because AutoResetEvent is expensive. The performance is slightly better but the main advantage is that it that the producer/consumer thread doesn't waste CPU while blocked in enqueue/dequeue .

I have also added a `stopWaiting()` method that unblocks the blocked consumer/producer .

I am attaching the new queue and testing script.