Grant Fritchey

12 April 2021

How Does Microservices Architecture Change Database Deployment?

This question was raised at the recent Redgate Summit: How does the implementation of a microservices architecture affect the implementation of a database DevOps approach?

I could even rephrase it a little: Does a microservices architecture affect a database DevOps approach?

Microservices is, or isn’t, one of the hot new trends, depending on how you look at it. The concept of a microservices architecture doesn’t hit the lists of hot topics. However, edge computing certainly does. So more people will be adopting the idea of tiny services, whether as edge computing, or as microservices. Some of those services have data needs. The question quickly becomes then, can a DevOps approach to database automation enhance microservices, or, is it completely replaced by the microservices architecture, or does it fall in the mucky middle somewhere?

Let’s talk about all three possible scenarios:

• Microservices supplants the need for database deployment automation
• Microservices architecture has some affect on how you automate database deployment
• A data store in Microservices is pretty much a data store any where else

Microservices Supplants Database Automation

Let’s assume we’re not talking about microservices or edge services that don’t use data at all. Of course, if you’re not storing information in any way, automating the deployment of data structures is an utter waste of time. Instead, let’s focus on those that gather, and store, data, no matter how temporarily.

I can easily make the argument that you have very little need for any kind of database deployment as such. The combination of possible technologies to define a microservice is probably infinite. So, let’s narrow things down a little. Let’s say you have a microservice that works to gather telemetric data from smart microwave ovens. Let’s assume a data store that is document based, like MongoDB or something along those lines. In all likelihood, we only retain momentary data at the collection point. Further, the need for any kind of data validation or confirmation is practically zero. With all this in mind, a database deployment is simply a matter or replacing what’s there. The need for any kind of specialized processing is just not there.

In short, sure, depending on the scenario, microservices makes it so that automating databases just isn’t necessary.

It Depends

As we saw in the first example, it’s easy to envision a place where the nature of the service and the nature of the data combines to make it possible to alleviate the need for some kind of database deployment automation. However, as was noted above, and as I’ll reiterate here, it does depend on the nature of the data being collected. It’s also partly a question of how long that data has to live. Further, the data itself is going to drive us to one structure over another, and the structure of the data in question drives a lot of other decisions.

Taking all of this together, the argument becomes one that is much more nuanced. It’s not a question of microservices vs. not microservices to determine when, how, or if, we’re going to automate database deployment. Instead, we immediately start looking at the data, how it’s captured, how it’s consumed, how long it lives, and more, to make the determination as to what we’re going to have to do to manage that data.

In short, a data store is a data store.

A Data Store is a Data Store

Let’s go back to our example of the smart microwave ovens. We decided that the telemetry information is better stored in a document database. What if, on further reflection, we decide that, in fact, it might be better to define a more rigid structure and go with a relational data store like PostgreSQL. Further, we are told by the business that the data collected needs to be as clean as possible at the point of collection. So now, we’re looking at tables, constraints, and possibly lookup data. Now, how likely is it that we’re going to need to come up with some automated way to deploy the database changes. In fact, since we’re talking about thousands, maybe tens of thousands, or even millions, of databases around the world, doesn’t it necessitate a rock solid automation methodology?

There are two issues with data that drive so many decisions around it. First, and probably most important, data needs to persist. Because we want to keep the data, a deployment mechanism that simply throws it away isn’t in our best interest. No matter how we do it, we’re going to have to deploy changes in a way that preserve the data we need to preserve. This single requirement drives us to automated solutions for database deployment. However, there’s a second reason.

Another massive factor that drives decisions on how we design, build and deploy databases is data cleanliness. You can create a system that collects any information from any source and stores it all in one place. However, reporting on that, ensure that meaningful queries can be run and that they have meaningful results, all that is actually a pretty massive undertaking. As soon as we determine we’re not in a place where either we just don’t care about all the data, and can therefore lose some, or, that there’s no way to put a structure on the data, or the data is simply changing too fast, in short, as soon as we determine we need clean data, our processes change. Another term for this is data integrity, and it’s an idea that relational data stores, of all types, have been dealing with for decades.

Conclusion

The real truth is, there is no hard and fast answer. We cannot simply say that microservices eliminates the need for automated database deployments. Instead, we find ourselves almost exactly where we are when looking at macro-services. What kind of data is it? What kind of integrity will we require? How long will the data stick around? How much of it do we have to keep? These questions, and more, drive us to choosing certain data management systems. As we pick those data management systems, we’re also presented with the need to be able to make changes to those systems in a fast, safe manner. That leads us back around to database DevOps.

For more reading, we have a whole slew of articles on SimpleTalk related to microservices. There are several great articles there, but I especially love this one by Matthew Skelton on how microservices affect the job of the DBA, specifically around automating databases & database deployments.