Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.
Projections in Event Sourcing are a way to derive the current state from an event stream. This can be done asynchronously as events are persisted to an event stream which can update a projection. You don’t need to replay all the events in an event stream to get to the current state for a UI every time you need to display data. This could be very inefficient if you have a lot of events in a stream. Rather, create a projection that represents the current state and keep it updated as events occur.
YouTube
Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.
Projections in Event Sourcing
Projections are a primary pattern you’ll likely use with Event Sourcing. It’s the answer to the most common question I get about event sourcing which is: How do you build out UI? If you have to replay all the events in an event stream to get to the current state, isn’t that very inefficient when it comes to displaying in a UI or reporting?
The answer, depending on your situation, could be yes. It could be very inefficient to have to replay an entire event stream to get to the current state.
This is where projections come in.
Data Transformation
Really a projection is transforming an event stream into another model. That other model could be almost anything depending on the events in your stream.
This is a stream of events for a specific product in a warehouse. Our event streams are per unique aggregate. In this case it’s for the product that’s identified with a SKU of ABC123
We can turn these series of events into a model that could be used for display purposes. The most obvious is probably to show users the current quantity on hand.
If we process these events we can derive these events into a current state that looks like this:
Our current state for the quantity is 59. If we process each event and keep track of the quantity (10 + 5 – 6 + 50) we would come to this final state.
The beauty of event sourcing is that you can create many different models. For example, we could also derive the event stream into this state:
In the above, we’re simply breaking out by keeping track of Received, Shipped, and Adjusted all separately.
As another example, we could keep track of product aging. Meaning how long is the oldest product in the warehouse from when it was received.
Event Consumers
Now before we get to actually building projections, you need to deliver/publish events to consumers that will process those events to update their projection of the current state.
There are a couple ways to accomplish this.
The first is to simply use a message broker that publishes events after they are saved to the event stream.
If you’re crossing a boundary, you likely don’t want to expose the event your persisting to your event stream. That would be leaking data internal to your domain. You’d likely want to transform that event into an integration event that you have versioning and contracts defined for.
The second option is if your consumers are within your boundary, and your database/event store supports it, is to use the event stream directly.
Products like EventStoreDB support two types of subscriptions: Persistent and Catch-up.
Subscriptions
Persistent subscriptions mean that you have competing consumers to a subscription group. As events occur, the event will be published to one consumer in the group that will process the message. This is similar to how you would use a message broker, except the event store is the broker. The event store keeps track of which subscription group has processed which message in the stream.
Catch-up subscriptions work a bit differently as the consumer must ask the event store to send events from a particular version onwards. The consumer must keep track of which message (version) of the stream it has processed. Once the consumer is caught up and processed all the messages that have occurred since the one it requested, the event store will send new messages to the consumer. Again, the consumer must keep track of which index/version it has processed because once it re-connects (for whatever reason) it needs to tell the event store where to start in the event stream.
Source Code
Developer-level members of my CodeOpinion YouTube channel get access to the full source in this post available in a git repo. Check out the membership for more info.
Building Projections
For my example, I’m using the first example I showed with a project that is keeping track of the current quantity for a product (by SKU).
I’m using Entity Framework and here’s my simple Entity and DbContext.
Every time a new event is appended to our event stream, it will publish this event which will be consumed by our ProjectionBuilder.ReceiveEvent()
ReceiveEvent() will determine which event type it is, then call the appropriate Apply() method. Each Apply() method fetches the appropriate record from our database, then updates the appropriate property/column. Then of course save the changes.
Now if we wanted to display the quantity on hand for a product, we would simply query DbContext by SKU and have to Subtract the Shipped from the Received amount. We do not need to go to the event store, reply to all the events to get to the current state. We already have it.
Demo App
I’ve created a simple console application that has all the code above in an interactive way. Developer-level members of my CodeOpinion YouTube channel get access to the full source and demo available in a git repo. Check out the membership for more info.