Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.
What’s the biggest scam in tech that is deemed acceptable? Best practices. Everything has trade-offs and your context matters. Let me give various examples to get out of this dogma about best practices.
YouTube
Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.
Best Practices
The topic for this video/came up when I came across this tweet from David Fowler.
I couldn’t agree more. Too often, best practices, principles, practices, patterns, etc., are treated as rules without considering the reasons they exist or how they apply in your context.
To illustrate this, let’s look at an older ASP.NET Core template example:
If you’re unfamiliar with the above code, don’t worry. Here’s the gist.
The Program class is the entry point when our process starts. It creates the default web host and tells it to use the Startup class.
The Startup class defines all of our dependencies used in the ConfigureServices method, and then the Configure method defines all the middleware that an HTTP request will route through.
Now let’s look at a newer version that consolidates this.
This new version is much more condensed and does away with the Startup. It’s more procedurally done but still specifies the types to be registered for DI and all the middleware. It’s ultimately doing the same thing as the prior version.
When this new template came out, there were a lot of reactions similar to this:
If you follow any principle as the law without being pragmatic, you’ll land in more complexity than you started with.
The Startup class just added indirection, for what reason? So it could separate registering types and middleware? What benefit does that have? In the example above, none in my opinion. The Startup class is being called by the ASP.NET Core Runtime. That’s it. If however, in your context you were registering types for DI for ASP.NET Core and possibly some other ServiceCollection used in a separate project that was shared, then sure, it could make sense.
The template is just an illustration. It’s not a best practice.
Example
Here’s another example in the world of messaging. My post/video about McDonald’s Journey to Event-Driven Architecture, it shows how they were using DynamoDB as a fallback when they couldn’t publish to Kafka.
This allows them not to lose messages that need to get published on a topic in Kafka. Separately there is an AWS lambda that pulls from DynamoDB and then attempts to publish to the Kafka Topic. It’s basically a retry mechanism.
Now if you were to listen to “best practices”, you’d likely find out that a common solution is to use the Transactional Outbox (Outbox Pattern).
Using an outbox means that you persist outgoing events that should be published to your primary database along with your business data within the same transaction. This way, you always reliably persist your business data and outbound events in an atomic operation. Separately you have a process that pulls the events from the “outbox” in your primary database and publishes them to your broker.
So does McDonalds have a bunch of terrible developers because they aren’t following a “best practice”. No. Obviously not. There’s tradeoffs of course.
With the outbox pattern, you could be adding a significant amount of additional writes to your primary database. Also depending on process that publishes events, it could also add more load because of queries to the outbox. Regardless, you are absolutely going to add more load to your primary database.
Is this a big deal in your context? If it is, maybe the outbox pattern isn’t a great idea.
If that’s the case, your option could be to use a fallback of some other durable storage, in their case DynamoDB. Are you guaranteed not to lose events? No, because DynamoDB could also be unavailable when Kafka was unavailable due to network issues. But that may be a trade-off they are willing to accept.
Context is King
There’s also a constant comparison of large companies in a positive or negative way. Amazon is often the example used.
Well Amazon does it, so we should too.
I think Microservices was an example of this where large organizations were using Microservices as a way to decompose a large system because of organizational constraints. Individual small teams owning services. Then microservices started getting adopted by a small team within a small company where they have 20 microservices with 3 developers.
There’s also the flip side, where the comment is
We’re not Amazon so I don’t need to do that
But everything lies somewhere in the middle. Absolutely you’re not Amazon, but there likely are things to be learned that can apply to your context.
Context is King.
Your context matters in determining what is appropriate based on your architecture, design, team, etc. Not everything is a “best practice” to you.
I wish we could as an industry rename “best practices” into “maybe this a good idea if you have these set of constraints, then you should think about it”.
Join!
Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out my Patreon or YouTube Membership for more info.