From time to time, I meet people that claim they do microservices. Since I’m interested to hear more real world experiences, I dig deeper by asking technical questions. And pretty often I learn that even though they have multiple microservices, they all use the same database.
Using the same database is a microservice anti-pattern. One of the advantages of the microservice architecture is decoupling the codebase and database to allow individual teams to move faster. Sharing a database between microservices basically kills that advantage.
Microservices are not free. They add complexity. That’s a downside. But that complexity is worth it because the benefits outweigh the downsides.
When you do microservices without separating the database, you get the complexity without that much benefits. And that complexity basically becomes unnecessary complexity. That kinda kills the whole purpose.
Note that I mean not sharing the same database — not database cluster. You don’t necessarily have to set up a separate database cluster for each microservice, but you have to use a separate database per microservice even if you’re sharing a database cluster.
What I mean here is that if multiple microservices use, say, PostgreSQL, what you could do is spin up a single PostgreSQL cluster but create a separate database in the cluster for each microservice.
And if you actually do use PostgreSQL, it has to be a separate database — not a separate schema — because you can do queries and joins across multiple schemas in PostgreSQL. If you go with a separate schema per microservice, it becomes too tempting to just fetch or join data of another microservice directly using its schema instead of going through its API. (Yeah, it is possible to prevent access to other schemas but then what’s the point of using schemas in the first place?)
The same applies to big data database systems like Cassandra. You don’t necessarily have to spin up a whole cluster for each microservice. Especially if it’s cost prohibitive. You could just spin up a single cluster and use a separate keyspace for each microservice.
Separate Database Clusters
Having said all that though, if you can afford it, it’s still the best to actually spin up a separate database cluster for each microservice. The reason to do that is the same reason microservices allow separate teams to evolve their microservices on their own pace without being slowed down by other teams.
Let’s say you have 20 microservices using one Cassandra cluster. Then a new backward incompatible version of Cassandra comes out. And it has a new feature that 2 microservices could take a huge advantage of.
In addition to that, imagine that 5 of those microservices are not being worked on anymore. They’ve been completed and work perfectly and no one wants to go and mess with their stability.
What do you do now? Do you force everyone to upgrade or force everyone to stay stuck on that version of Cassandra forever?
If you force everyone to upgrade, it has to be a coordinated effort involving multiple teams to upgrade and test all 20 microservices to ensure nothing breaks.
If you force everyone to stay on the current version of Cassandra, some teams won’t be able to take advantage of the features of the latest version.
Even if the latest Cassandra release is backward compatible, it’s still a risk to upgrade because, say, new bugs get introduced. Even if Cassandra developers meant the release to be completely backward compatible, sometimes it just doesn’t happen specifically because of some bug that unintentionally breaks backward compatibility.
It’s too much stress and effort to upgrade a database cluster used by many microservices. And because of that reason alone, the cluster might get stuck on an older version forever. And that kills another advantage of going with microservices — using the latest technology.
Another reason to separate database clusters is that in high load situations database systems have to be tuned for a particular workload. Depending on the type of the load, a database cluster might have to be optimized on CPU, RAM, or disk.
If microservices sharing the same cluster have different types of load, it might be impossible to optimize the cluster to satisfy the needs of all microservices. And being able to do that is another advantage of going with microservices.
All those reasons is why it’s best to go with a separate cluster for each microservice. If you have budget for that, of course.
Staying on Budget
Since starting with a separate cluster for each microservice might be cost prohibitive for some companies, the middle ground between sharing a single cluster or going with separate clusters is starting with a single cluster and then move the microservices that need newer versions of the database system or have a specific type of workload to a separate cluster when that need arises.
In that case, those 2 microservices that could take advantage of the latest version of Cassandra, could just spin up another cluster with that version and migrate to it. Both microservices could then share it.
Newly developed microservices would start with this newer cluster as well.
And older microservices could upgrade and switch to the newest cluster at their own pace without having to coordinate the effort with other teams.
Eventually, some of these microservices will stabilize and a new version of Cassandra will get releases again. Then the cycle could get repeated with spinning up a new cluster and repeating the whole process.
Or when a particular microservice becomes high load and requires a specific type of database system optimization, a new cluster satisfying those needs could be spun up and the microservice migrated to it.
This way, you can get the best of both worlds — let microservices evolve independently and at the same time don’t spend money on additional clusters unless you really have to.
Sounds like a win-win to me.
Sharing databases between microservices nullifies many advantages a microservices architecture brings to the table.
At the very least you should use separate databases even if they’re hosted on the same database cluster.
Ideally, you should have a separate database cluster per microservice. That’s when you can really rip the benefits of going with microservices.