Creating a scalable message system in C# – Part 1 (Requirements / Background)

This is the story all about how.. there was a requirement at my last job to create a message system to essentially migrate our platform from a polling / cron style of work to a real time system.

Some background on the previous system as it was:
We made use of Hangfire as a job / worker system to handle tasks such as releasing orders through the system, emailing customers, and doing a wide range of tasks. The way in which these were enqueued was that we would have a Scheduler job run through once ever 5 mins and enqueue jobs for everything that had changed so had something that needed to be done.

This system worked fairly well, however we really wanted to get things moving in real time – so we had the challenge now of trying to work out the best way to do this real time job creation.

As our jobs needed to be enqueued based on changes in a 3rd party software, as well as changes in our own custom software – all of which was running on MSSQL, we actually looked into an decided upon the use of a fairly unused (from what I’ve seen) part of SQL Server which is their Message Broker system, a built in Message Queue system within SQL.

At this point we now had a rough outline of the project:
1. We needed it to be something we can scale – we were thinking horizontal scaling on this essentially, being able to run the app on multiple machines.
2. We needed it to work based on SQL Server Message Broker to allow us to interact with 3rd party and proprietary changes.
3. We needed it to based in C# (our tech stack)
4. We needed it to be easy to configure and get running, so that we can quickly and easily expand out the application.

After some initial research, we came across an Open Source project which met some of our needs and would be a pretty ideal starting point – https://github.com/dyatchenko/ServiceBrokerListener – This project was a basic system for working with Service Broker in MSSQL.

Now this open source code had some things we would need to change straight away, It was based around code in an old .Net, which we wanted to update, it had some Threading issues which would need fixing and it was based around cleaning up all of the Sql side of things whenever the app closed, which is something we really needed to change.

Our system needed to ensure that nothing would get deleted or missed when we didn’t intend it to, so we needed any automatic cleanup to be gone, and moved into a way to handle it manually.

With this all in mind, it was my task to work on overhauling the open source project to fit our needs, and coming up with an app though would allow us to scale this system out.

Part 2 – Re-Writing the open source code to match our requirements.