Redgate’s annual “State of the Database Landscape” survey has been published. Like every other year, it paints a really interesting picture. Personally, I love looking through this in order to better understand where people are experiencing pain in the management of their data. If you know where people are experiencing pain, as a technical person, you know where to focus your own skill development. As someone in charge of those technical people, you better understand where you’re likely to be investing time and money. 

So, the fact that there’s a real, growing, issue around the complexity that we find ourselves dealing with in our data management is more than simply noteworthy. Knowing that you’re not alone in dealing with the complexity provides at least some comfort. However, let’s drill down a little to better understand where that complexity is arising from, and how best to deal with it.  

Migrating Your Data 

There was a time, not all that long ago, when the majority of organizations only had one or two data platforms within the organization. This year, that number has shrunk to just 16% of the respondents: 

As someone personally learning multiple database platforms, any myth of simple interchangeability between, for example, Oracle and PostgreSQL, or any other set of data platforms, well, it’s a myth. However, one part of the growth of platforms we have to manage, is a wish to move between platforms. Your organization could be moving off of, for example, SQL Server, because they’ve found the combination of PostgreSQL and postGIS to give them a competitive advantage, so the migration between platforms start. Alternatively, maybe your organization has decided they no longer wish to pay Oracle licensing fees and they’re moving to PostgreSQL. Regardless of the reason or the platforms, you’re moving your data. 

However, what’s quickly discovered is that, of course, you’re also moving your code. You don’t connect to two different data platforms in the same manner. The query mechanisms between platforms of course differ. And you suddenly find that, sure, you’ve moved several databases, quickly, and successfully, between platforms. However, you’ve got some that are taking quite a bit longer. Others, that you’re figuring out, may never be migrated until they’re retired or replaced. Instead of migrating to a new platform, you’ve added to the number of platforms you have to manage. 

Funny enough, the other migration pattern that we’re seeing is the move to the cloud (and for some specifics about the move to the cloud, look to our detailed report on that from the State Of survey). Again, the decision for the cloud move could be technical, or financial. It doesn’t really matter why. You’re now migrating data to the cloud. Maybe it’s on an identical platform. You’re running SQL Server on virtual machines locally. You’re moving to a cloud-hosted VM. You’ve decided to get rid of the local machines running PostgreSQL and instead host it in AWS Aurora. Well, the same issue occurs. A few databases get moved very quickly. Now some are kind of bogged down. We’re now running a hybrid environment, some in the cloud, some locally.  

All of this, all of it, makes for much more complex environments. Because the complexity has increased, teams are going to struggle. You’ll certainly see some pushback on the release of new functionality cadence. Your teams might struggle to learn how to appropriately manage the new technologies they’re working with. Then there’s the added overhead of possibly managing data in more than one location, being forced to migrate some from one platform to the next.  

On top of the complexity, your teams are without a doubt wrestling with ensuring that your multi-platform, hybrid, possibly even multi-cloud, environments are secure. Protecting data has become more and more important. We’ve all realized that our unique data is a big part of what drives our organization. Losing control of that data has repercussions. So now, your teams are dealing with making sure that they can secure everything, whether they know it well or not, to the same degree.  

Finally, all of this is coming before many of us are ready. We’re still doing manual processes to manage and deploy our data. We’re taking existing poor processes and now, they’ve got to run well between three, or more, different data platforms as well as our local servers and the data in the cloud. All of this, all at once. 

Further, with the takeoff of Agentic AI, all of the issues we have around our automation is further exposed with the speed at which new code and new functionality are generated. Teams are being forced to pause some work while they learn how to automate their testing and deployments. 

I want to pause here a moment to note, what I’m describing are the challenges creating the rebound caused by complexity. None of this is suggesting that we shouldn’t migrate our database. There are extremely valid reasons to set out on that path. However, we have to know that the path is not without its dangers. That doesn’t mean there’s an excuse to panic, or try to hit the brakes on the progress these migrations represent. 

Don’t Panic 

Instead of panicking, we should simply get educated. Here’s how we’ll go about it: 

•  First, take advantage of all the information the survey gives us to focus our learning. We know where our peers are being challenged, even before we have to start our own migrations.  

• Use that information to make an honest assessment of your team. Where do you have weaknesses that you can shore up through education or bringing on a new person? This is much more easily addressed when you’ve made that assessment to form a base of knowledge. More importantly, where do you have strengths? Your strengths can be reinforced through these kinds of migrations. Knowing where you’ve already got a great team and solid processes gives you a foundation to more successfully make that migration.  

• Finally, be willing to accept that, in fact, maybe not every single database can be moved between platforms.  

So, yeah, I really love the State of the Database Landscape Survey, because I do like to have a better understanding of where I should focus my own learning. Forewarned is forearmed.