Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.
Message driven architecture doesn’t need to apply just to Microservices. You can apply a Message Driven Architecture to a Monolith to decouple concerns. Consuming Commands and Events to handle concerns into their isolated units. The asynchrony and isolation will also enable your application to be more reliable and resilient.
YouTube
Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.
Monolith
Just because you’re developing a monolith, doesn’t mean you can’t leverage a message driven architecture. Messaging isn’t simply about communicating with other services. You can use messaging to communicate with your own monolith.
Typically we think of a monolith as a single application that’s singularly deployed (maybe many instances) that interacts with a database. Yes, there may be some other resources such as a cache or 3rd party APIs but for the most part, people think of it as a simple diagram:
There are two different aspects to a message driven architecture we can use to decouple various concerns within our system. Commands and Events, which are both types of Messages. The terms Producer and Consumer are often used. A producer is creating messages and sending them to the message broker while the consumer is receiving and processing the messages. In the case of a monolith, it will be both the producer and the consumer.
One thing to be aware of is that using a message broker is moving work to be asynchronous. Meaning you’re pushing work that was previously all done together to now be done separately in isolation. This isolation is what gives you resiliency and reliability to your system, however, it comes at the cost of a mind shift of being able to realize work that can be done asynchronously.
Publish/Subscribe to Events
First, the publish/subscribe pattern is when the publisher publishes an event to a message broker. The publisher is totally unaware if there are any consumers. There may be no consumers, there may be many.
Above we’ve published a message to the message broker from our monolith. If we had 3 consumers that want to process that event, that message would be delivered back to our monolith to all three consumers within it.
To illustrate the utility of this within a monolith, let’s say you have an e-commerce site. When a customer places an order, you must save the data to your database as well as send the customer a confirmation email.
Saving the data and sending the email are two different concerns. If these are done together in the same process, there are various ways this could fail.
What if the order is saved, but the email service is unavailable? Will the email never get sent? Will the customer see an unexpected error? Sure you could wrap these calls in retries as well as handle the exceptions, but you’re adding a lot of complexity to handle faults simply because you have two concerns that are mixed.
In this case, when the customer places an order, you would publish an OrderPlacedEvent to the message broker. You would then asynchronously be consuming that event back in your monolith, which would be processing that event to send the email.
In this case, if your email service is unavailable, the order will still be persisted, however, the email will not get sent. Most messaging libraries support ways to manage retries upon failures, exponential backoffs, and moving messages to a dead letter queue. Check out my post on Handling Failures in Message Driven Architecture.
In many transient failures often a retry will be successful. This means that if the consumer failed to process the message it will try again and likely succeed. In the case of our email service, if it had a transient fault, once we retry it may be successful. In this case, we never lost the order, the customer didn’t see an unexpected error, and the confirmation email was eventually sent.
Enqueue Commands
Another way we can leverage messaging is by sending commands to a queue. This is different from publish/subscribe as every command has one single consumer.
We can leverage a queue by creating commands for various work that can be done asynchronously. As mentioned earlier, the shift is moving from a synchronous workflow to an asynchronous one can bit a bit of a mind shift. As an example, when a customer is placing their order on our site, do we really need to persist it to the database? What if we simply created a command that we send to the message broker/queue and then immediately return a response back to the customer that their order was placed?
If we have a high-traffic site and our database gets overwhelmed, we won’t lose any orders as customers are placing them, nor will our customers see any unexpected/database errors. We are simply creating messages and sending them to the queue to be processed.
The isolation of consuming messages gives us reliability and resilience but it also allows us to scale. Check out my post on the Competing Consumers Pattern for Scalability
Asyncrony of Message Driven Architecture
The biggest concern with messaging is understanding where and when you can move work to be asynchronous. If a user clicks a button and expects a specific result to be returned back, that can only be done if the work is synchronous. However, there are many instances where this comes down to expectations.
For example, if you are in a cloud provider’s portal and stop a virtual machine. Their UI will update saying the virtual machine is stopping. It didn’t instantly stop. They are using the same methodology of queuing work to stop the virtual machine but make you the end-user aware that it’s in progress and not fully complete.
Understand where you can move work to be asynchronous and what if you need to set users’ expectations about when it occurs.
Source Code
The YouTube video that covers this post shows actual code and a demo application. Developer-level members of my CodeOpinion YouTube channel get access to the full source for any working demo application that I post on my blog or YouTube. Check out the membership for more info.