Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.
Can we, as application developers, stop over-engineering software? I hate to use the term engineering even to describe it! I’m guilty of it too. I was writing “clever” code that was overly complex, hard to understand, and hard to change. Here are some of the pitfalls I see and what I think about to guide me down a more straightforward path.
YouTube
Check out my YouTube channel, where I post all kinds of content accompanying my posts, including this video showing everything in this post.
“What if” Game
One of the traps I think developers can get into is playing the “what if” game. When we, as developers, gain more insights into the domain we’re working in and start making assumptions. The issue is then we don’t validate those assumptions, which can be widely incorrect. Often this occurs when we discover an edge case we think we need to solve within code. In reality, it’s so infrequent that the business doesn’t need it to be automated or handled by our code. Instead, it could produce some notification or indication to someone in the business, and they can manually resolve it. Not everything needs to be handled in code, especially when the solution is complex and the case of it occurring is very low. This is just a low-value, high-cost edge case. Validate assumptions, and talk with the business! Don’t write complex code when you don’t need to. Stop over-engineering.
Caring too much about tech
Developers love everything code, especially new and shiny libraries, frameworks, and tools. We can lose sight of why we’re writing code in the first place. It’s to solve business problems and create value. Yes, we want to stay up-to-date with the latest technology so we can improve our systems. However, we have to have an equal amount of focus on the domain we’re working in.
Care about the domain as much as the tech.
We’re writing software to solve business problems, and you need to understand your domain. If all you care about is the latest tech, you’ll be adding libraries, frameworks, and tools that are not required, and only making the solution more complex is just over-engineering. Use the tech that best solves the problems that you have. Stop chasing the new and shiny if it doesn’t provide any value to your system.
Don’t roll your own!
It’s unlikely that you’ll genuinely need to write your own library or framework. But how often have you worked in a system with its own web framework, ORM, etc.? Likely you have. Was it documented? Absolutely not. It was some homegrown framework that someone who is no longer at the company wrote years ago, and now you’re stuck with it. I get it; we love tech but all your adding is complexity when you go down the road of writing some fundamental framework that your system depends on.
A good example is writing your messaging library on top of RabbitMQ/AWS SQS/Azure ServiceBus. People often do this without realizing all of the different patterns and concepts they will need to implement, and before long, they have their own homegrown messaging library. Instead, you could use a battle and production-tested messaging library that has all the features & patterns that are commonly required with messaging. Buy, don’t build complexity by over-engineering.
Not Enough Up-Front Design
Not understanding the domain enough will lead to making an overly complex system, likely because of over-engineering. How? This relates to the “what-if” game. Developers will often add layers of abstraction and useless indirection because of “what-ifs.”
“What if we need to do X” or “What if X changes, let’s add Y.”
YAGNI (You aren’t gonna need it). Adding useless layers of abstraction indirection that you’ll never need.
I’m not talking about BIG upfront design in a waterfall-style specification. But exploring the domain, especially in a green-field project by leveraging something lightweight like event-storming. Understand the workflows and the different perspectives of different people within the business.
DRY per Boundary (be explicit)
Don’t repeat yourself has caused a ton of confusion to developers. A great example of DRY gone wrong is when things get overly generic trying to accommodate different use cases, ultimately leading to a ton of complexity.
Entity Services are another example of DRY gone wrong. Entity services are what I call services that operate on a single entity or a set of entities. And that entity only ever exists within that boundary. An example of this is a ProductService in a warehouse system. A ProductService contains all the product information, such as the name, price, cost, quantity on hand, etc. and all the various ways to change that data.
The problem with this is that you do want to repeat yourself. The concept of a product in a warehouse exists in multiple different boundaries. Sales would care about the price; Purchasing would care about the cost; Shipping & Receiving handle the quantity on hand. There is no single Product Entity, but that rather each boundary would have its own concept of a Product exposing the functionality and data that is relevant to it.
Over Abstraction
It seems pretty typical to want to abstract any 3rd party dependency, library or tool. On the surface, this makes sense, and you create your own abstraction so that your application code depends on it rather than the 3rd party. That way, if you ever need to change that dependency, you don’t have to change all your application code, you change the abstraction you created. Sounds reasonable? Not entirely, it can be over-engineering.
It comes down to coupling and managing it. The point of the abstraction is to simplify the underlying concepts best suited for your use case. Creating an abstraction will limit your ability to leverage all the dependency has to offer. Manage coupling! How? Vertical slices and focus on features. Grouping functionality by features allows you to narrow your focus and make localized decisions about what you depend on per feature (or feature set).
Join!
Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design and access to source code for any working demo application I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.