The State of the Database Landscape

The 2024 industry report from Redgate includes data from every sector and every company size, from developers, DBAs and software delivery professionals to IT leaders, CTOs and CEOs across the world.

Methodology

  • 3849 respondents
  • 6 continents
  • 15 industry sectors

AI (Artificial Intelligence)

Emerging technologies and barriers to implementation

The past twelve months have seen incredible advances in AI technology and machine learning tools, effectively changing the way systems and professionals can work. From creating content, generating code and producing images, to analyzing information and curating presentations, AI is the biggest transformation in the digital world that the industry has ever seen (not including the invention of the internet, which is of course the source of all things digital).

Incorporating these tools into technology stacks, however, isn’t straightforward. AI is new, its possibilities and limitations have yet to be explored fully, and developers are playing catch up in terms of the knowledge, skills and consequent training required to make the most of it.

Methodology

AI with Jeff Foster, Director of Technology and Innovation, Redgate

AI with Jeff Foster, Director of Technology and Innovation, Redgate

Determining the AI lifecycle and user expectations

As with any technology, each tool starts at the beginning of the adoption lifecycle; from early innovators and adopters, through to the late majority and laggards. AI is much hyped with many having high expectations of its potential. As evidenced in our survey, people are indicating that the value of AI is recognized, with 20% of respondents answering they had used AI technology in the context of database management and 35% stating they are considering adopting it in the near future. 

It’s not a silver bullet, and you need to have the fundamentals in place already – decent code reviews and the right kind of organisational culture. AI still needs human oversight and process-based guardrails to help prevent the buildup of technical debt.Jeff Foster

Jeff Foster

Director of Technology and Innovation, Redgate

Has your organization used AI in the context of database management?

  • Yes, we have (20%)
  • No, but we plan to (35%)
  • No, and we have no plans to (32%)
  • Other / N/A (13%)

Early AI adopters and implementation

There are a multitude of potential uses for AI with regards to database management, with analyzing and optimizing database queries or code being top of list. This aligns with our survey data where 64% of those already using AI said they had used it for query and code optimization.

Beyond that, 65% are using AI tools to help with tasks involving database schema, and 55% to generate sample data or code snippets. Along with simulating various database scenarios and automating testing scenarios, it appears that AI has already found a natural fit in the development and testing of databases.

Has your company used AI for testing/development purposes?

Testing and development tasks involving database schema
  • No (45)
  • Yes (186)
Analyzing and optimizing database queries or code
  • No (66)
  • Yes (151)
Generating sample data or code snippets
  • No (58)
  • Yes (147)
Validating and analyzing database changes during development
  • No (144)
  • Yes (70)
Simulating various database scenarios for testing purposes
  • No (64)
  • Yes (142)
Automating testing scenarios
  • No (69)
  • Yes (129)

Beneficial outcomes of AI

AI tools, when used correctly, have many benefits for database professionals. The promise of making tasks quicker and easier being the main draw, as well as error detection and correction.

The survey asked respondents who have been using AI what were the main beneficial outcomes they experienced, and being more efficient via streamlining and automating tasks came top. Enhanced security and standardization followed closely after and optimizing database performance and gaining advanced insights coming behind. The results suggest that with correct application, AI can deliver on every element.

The main benefits organizations have experienced from using AI

  • Streamlining tasks / efficiency (40%)
  • Automation (40%)
  • Enhanced security (39%)
  • Standardization (39%)
  • Optimized database performance (34%)
  • Advanced insights (27%)
  • None (6%)

The potential uses of AI within database management

Aside from respondents who had already used AI, those who were planning to in the future were asked what kind of tasks they were interested in adopting AI for. The most popular task selected was optimizing queries and generating and reviewing code, which are obvious candidates for AI tools because, by their very nature, the tools can be trained on a vast repository of queries and code already written.

Other uses now being looked at in the database space are predictive analytics, data modelling and schema design, anomaly detection and security, and automating database management.

The top 10 tasks organizations are interested in using AI for

  • Query optimization (54%)
  • Code review and generation (48%)
  • Predictive analytics (40%)
  • Data modelling and schema design (37%)
  • Anomaly detection and security (33%)
  • Data quality assurance (32%)
  • Automating database management (29%)
  • Natural language interfaces (26%)
  • Data indexing and search (25%)
  • Data backup and recovery (20%)

Popular database development AI tools

As with any new digital development, it doesn’t take long before a vast number of tools appear on the market. The same can be said for AI, which although it is still relatively new, already has a wide range of tools emerging, helping with a variety of tasks.

However, in terms of database and development there were five particularly popular tools cited in the survey results as already being in use. 39% of respondents are using GitHub Copilot, 26% are using Amazon CodeGuru and CodeWhisperer, 23% are using DeepCode, and 21% are using Tabnine.

The top 10 AI tools used by organizations

  • GitHub Copilot (39%)
  • Amazon CodeGuru / CodeWhisperer (25%)
  • Otter.ai (24%)
  • DeepCode (23%)
  • Tabnine (21%)
  • Scribe (20%)
  • Quillbot or Wordtune (19%)
  • Mintlify (17%)
  • AskCodi (14%)
  • CodeWP (14%)

The risks of using AI with the database

While there are advantages to be gained from AI, three main areas of concern also emerged in the survey. The biggest is the practicality of actually using AI. Data security is of concern, in particular how a chatbot like ChatGPT will use any data entered into it. There then comes the accuracy of the results generated and the training and expertise necessary to write the best prompts to achieve the best results.

The ethical concerns around the data stem from information being used to train the Large Language Models behind AI chatbots, and how that impacts regulatory compliance. 2024 will see the introduction of the Regulatory AI Data Act in Europe which directly addresses these concerns and will undoubtedly, set the precedent for future AI use.

The question of training and expertise in the area is likely to become more important over time as privacy and regulatory requirements are resolved. This includes the ongoing management and support of using AI tools, the performance overhead and integrating them into existing workflows.

The top 5 concerns with using AI

  • Data security and privacy (41%)
  • Accuracy and reliability (36%)
  • Training / expertise (28%)
  • Ethical concerns (27%)
  • Regulatory compliance (24%)