Mri Pandit

21 May 2024

What to consider when introducing a new database platform

With the rise in the quantity and the complexity of data organizations now need to handle, the nature of the data that you store is driving the reason for your database platform of choice. Sometimes you have a mix of data, where some is relational and some is heterogeneous, which cannot be contained in a relational format. So that’s why, as an organization, when you evolve your business, you have to switch to a different flavor of database, which is not relational.

For example, if you’re a mortgage company, you store a lot of documents for your customers and you might want to move to a document store database. Traditional relational databases like SQL Server or Oracle aren’t great at working with both image files and text files, so it’s better to opt for a document-oriented database that is better able to handle semi-structured data. The top four in the DB-Engines ranking are currently MongoDB, Databricks, Amazon DynamoDB and Microsoft Azure Cosmo DB, which you’re probably already familiar with.

You’ll have similar demands when it comes to unstructured, time-series, geospatial, graph and other kinds of data. You need to introduce a new database platform and there are many options out there. As of May 2024, DB-Engines ranks 420 systems, compared with 18 when its first ranking list was published in October 2012. As a result, it’s not a simple choice because there are lots of considerations that you need to play in your research. I’d suggest focusing on four key areas.

The four big issues

The first big driver is integration with your existing systems. How compatible is it with both your systems and your people? There are also the costs involved in adopting, working with and managing a new database, which can add up very quickly in unexpected areas. Then there’s the sometimes thorny issue of getting your developer teams onboard with the latest technologies and new ways of working. And, naturally, there’s the ROI – how will you measure the success of the new database platform?

Alongside the same kind of scalability, disaster recovery, and availability capabilities you expect from any database, those are the four major considerations you need to think about before you adopt any new technology in database platforms. So let’s look a little deeper.

1. The ease of integration

Integration with your existing infrastructure is important, particularly in terms of how much coding you have to do when parsing data to your upstream and downstream systems. You need to make sure that data loss is prevented, and you’re getting the same data that you’re expecting.

You also need to consider your current monitoring solutions, your solutions of ingestion of data processes, and your reporting systems. Will they integrate with the new technology or are you going to be building another platform? It’s critical, for example, that your current monitoring systems can also monitor this new platform. Otherwise, you’re investing in another tool cost, and how are you going to validate the same health checks established for your existing database?

Say you have an event streaming platform, for example, which is designed in Kafka. In that case, do you have Kafka connectors which can work with this database technology? You need to be able to assess that even before you make a determination that this is the technology to go with.

Beyond these infrastructure questions are your technological abilities in terms of the skills of your people. How quickly can they learn or adapt to this new technology? If you’re introducing something which is very unique, like when Cosmos DB was first introduced on Azure, not many people were familiar with it. If you’re going to be one of the first customers, then you are solely dependent on the vendor because you can find very few resources online early on. So are you okay with taking the initial adopter phase for a project with a super tight timeline? That’s a risk you sometimes need to consider as a team when trying to bring in a newer technology.

2. The real costs involved

Even with open source platforms, there are costs involved when introducing them which can add up very quickly. If it involves a lot of developer time in the setup process, for example, then you’re adding additional costs along with any licensing costs. And it’s also going to be an overhead for you if you’re bringing in something which is new, and which you haven’t tried and tested before.

You also need to consider the ease of usage. Having the skillset within your people who already know the technology is part of it. Or how much upfront cost will it take? Are you looking as a hiring manager to bring in this skillset? Are you going for contracting or are you going to hire full time people to manage this new platform? People are expensive, with the US Bureau of Labor Statistics reporting the median annual wage of software developers as $132,270 in May 2023.

So it’s not just the platform cost itself, it’s also the managing the codebase cost, the skillset cost, and it’s the managing the people cost.

3. Getting your team onboard

So you’ve considered the ease of integration and calculated all of the costs. What about your current team? How do you get them invested in bringing on a new platform?

One way is getting the team introduced to training with the new platform. Most vendors are okay to roll out new tenant training opportunities for your existing team, and sometimes you can have instructor-based training as well. Then you need to get your team excited about the new technology, maybe by sharing a fact or a success story from some similar situation, like this company brought in this and this was the revenue turnover. Or you can share your own personal experiences because you have people with different skillsets who are in the team. Somebody might have worked on that technology already, and someone else may be familiar with it.

Leveraging personal examples within the team are the most beneficial ones because it’s easier to put the seeds of change inside and grow them organically. But I also feel that once they have some success initially with, let’s say, how they integrated this table and are now seeing data over here, they realize this is cool, so yes, they can do it. Smaller stories of success need to be celebrated, especially in the initial phases for bigger adoption to happen.

When you’re switching technology, I think it’s very important to also consider the upskill path for your existing team. How much do they know and how much effort will they be putting in? Like, will they be able to get familiar within a quarter versus a whole year to get there? This learning curve is a very significant consideration before you move into another technology.

4. The expected ROI

You also need to think about what kind of ROI would be expected. Is it cost savings, for example? Is it delivering new features faster, easier? Is it how much scaling can you achieve for this data?

Your ROI depends on your business needs and it varies from project to project and company to company. Some companies look at the dollar amount that it’s going to bring in. Some care about the higher upfront cost but they’re okay with adopting a technology as long as their systems are going to scale. So it’s a mix, it’s not a one way street that this is how the ROI is defined. It could be your DR readiness, your operational readiness, like how quickly can you bring in this technology and how quickly you can integrate it? So within three months, if you’re able to do it, then that’s ROI right up front.

There are different types of defining your ROI, so make sure to clarify what kind of ROI is expected, how it will be measured, and what the time frame will be.

Summary

Introducing a new database platform isn’t easy, but it can be easier provided you focus on the integration issues, the costs, team onboarding, and the ROI beforehand. There may be other issues as well which are particular to your own business or sector, like data security or the ability to scale very rapidly. Whatever they are, the assessment phase is where you should put in the work. That way, you’ll have fewer problems when it comes to the implementation phase.

This is a guest post from Mri Pandit, Senior Manager at Navy Federal Credit Union, the world’s biggest credit union with 13.5 million members. If you’d like to find out more about the new database landscape, watch the Redgate webinar: How to excel at managing multiple database platforms, featuring Mri, along with Joshua Higginbotham, Senior Architect at Centric Consulting, and Redgate Advocate Steve Jones.