Inside Perspectives: Database technology – 25 years and beyond
The relational database industry has changed a lot in 25 years. Find out how these changes have impacted the role of a DBA, and the industry as a whole, with Redgate Advocate Kellyn Gorman.
Over the past 25 years, the relational database industry has experienced a dramatic transformation. With Oracle, SQL Server, MySQL, and PostgreSQL as foundational technologies, these databases have evolved in response to the emergence of big data, NoSQL, cloud computing, and, most recently, AI. As a DBA, I’ve had a front-row seat to these changes, witnessing firsthand how the industry has shifted and how my role has adapted along the way.
Evolution of relational databases
When I began my career, relational databases were the backbone of data management, designed for structured data, for both transactional workloads and reporting. Oracle dominated enterprise environments, SQL Server thrived in Windows ecosystems, MySQL became popular in web applications, and even PostgreSQL’s predecessor, Ingres, was used from time to time for transactional workloads before open source became the driver it is today.
Big data and the rise of NoSQL
The explosion of data from social media, e-commerce, and internet of things pushed relational databases to their limits. Handling the sheer volume, variety, and velocity of data became a major concern, leading to the rise of NoSQL databases like MongoDB. These systems offered flexible schema designs and the ability to handle unstructured or semi-structured data, creating new possibilities for scalability.
Initially, NoSQL felt like a threat to the world of relational databases, and perhaps even a replacement for them. However, instead of being replaced, Oracle, SQL Server, MySQL, and PostgreSQL adapted. They began to incorporate support for semi-structured data, offering features like JSON and XML storage. Relational databases and NoSQL systems often ended up complementing each other, with hybrid environments becoming more common.
For me as a DBA, this was a pivotal moment. It became clear that understanding both relational and NoSQL technologies was crucial. I embraced the opportunity to learn, and my ability to design systems that could efficiently use both types of databases became a career advantage.
The cloud revolution
The rise of cloud computing such as AWS, Microsoft Azure, and Google Cloud introduced a new way of deploying and managing databases.
Tasks that once took up most of my time, like hardware provisioning, backups, and disaster recovery, became automated in cloud environments. Managed services such as Oracle Autonomous Database and Azure SQL Database promised self-managing capabilities, reducing the operational load on DBAs – but, as many of us discovered, only for the smaller workloads, and less often for the larger databases that consumed most of our allocated time. As these changes significantly removed overhead from our daily demands, we embraced them and became cloud specialists, most often within organizations running hybrid environments.
While this initially seemed like a threat to job security, I soon realized that it opened new opportunities. The focus of my work shifted from routine maintenance to more interesting work like database optimization, performance tuning, and designing scalable cloud architecture. The cloud also allowed me to explore global-scale disaster recovery and real-time analytics in ways that weren’t feasible on-premises.
AI’s emerging role
Today, artificial intelligence is transforming the database world once again. AI-driven optimizations in Oracle, SQL Server, PostgreSQL, and MySQL help automate tasks like query optimization, coding and indexing.
AI also presents new opportunities for integrating advanced analytics directly into databases. Machine learning models can now be embedded within databases to perform predictive analytics on real-time data. This development has changed how both DBAs and developers think about applications and data management.
For DBAs like me, AI is less of a threat and more of a partner, allowing us to focus on more strategic initiatives. Whether it’s tuning databases for performance or integrating AI-driven insights, the technology is helping me take on a more proactive and innovative role in managing data.
The changing roles of DBAs and developers
The evolution of database technologies has significantly changed the roles of both DBAs and developers. DBAs are no longer solely responsible for backups and patch management. Instead, we’ve become data architects, focusing on cloud migrations, security, and compliance. We’re now expected to have hybrid skills that span cloud infrastructure, AI, and data security.
Developers, on the other hand, have taken on more database management responsibilities, thanks to the rise of DevOps and DataOps. Automated deployment tools, containers, and CI/CD pipelines mean that developers now work more closely with DBAs to ensure that applications and databases are designed for scalability and performance from the very beginning.
As a DBA, these changes have been a net positive for my career. By embracing cloud and AI, I’ve been able to evolve into a more strategic role. Rather than simply managing databases, I now focus on optimizing them for modern applications, ensuring data security, and collaborating with developers and architects to design scalable systems.
Looking forward
The future of relational databases looks promising. With the continued rise of AI, the integration of cloud-native databases, and the convergence of relational and NoSQL systems, the role of DBAs and developers will keep evolving. For those willing to adapt and learn, this transformation offers endless opportunities to grow and innovate.
The last 25 years have been an incredible journey, and I’m excited about what’s next. As databases continue to evolve, so too does the potential for building smarter, more efficient systems that can leverage the full power of data. For DBAs, the future is bright, and the skills we’ve developed will continue to be critical in the years ahead.