The State of the Database Landscape

The 2024 industry report from Redgate includes data from every sector and every company size, from developers, DBAs and software delivery professionals to IT leaders, CTOs and CEOs across the world.

Methodology

  • 3849 respondents
  • 6 continents
  • 15 industry sectors

Security and Testing

Data is less secure in development and testing environments

The database landscape is changing – and changing fast. While the headlines are on the big topics like the rise in the use of the cloud and multiple database platforms, and the rapid adoption of AI, it appears that attention has turned away from the heart of the database: the data.

Data security measures are sporadic, the safety of personal data in development and testing environments is at risk, and two thirds of organizations have no individual or dedicated team who can be held to account.

Given the increase in the adoption of Database DevOps, this is disappointing. Database DevOps opens the door to releasing database changes faster with robust, reliable, repeatable processes. Conversely, they can also be safer because processes to protect data like provisioning development teams with masked and anonymized data can also be automated and streamlined. While data security may not be the goal, it can also be the prize.

Methodology

The data used in development and test environments

Copies of production databases are typically used by developers so they can test their proposed changes and ensure they work as expected. Those copies need to be realistic and truly representative of the original in order to make the testing accurate.

Results from the survey show that different IT teams have different ways of provisioning the data at different times. Respondents were asked to tick all of the options that apply and, while 47% use synthetic data, 43% use a full-size production backup and 28% use a subset of production data. This raises issues about the safety of sensitive data that may be in use in development and test environments which are less well protected and monitored than production environments, along with the lack of established processes.

The risks and rewards of using production data for testing

Which types of data do you provide for development/test environments?

  • Synthetic data (47%)
  • Full-size production backup (43%)
  • Subset of production data (28%)
  • Unsure (6%)

The approaches to handling sensitive data

With 43% of respondents using a full-size production database backup for development and testing, and 28% using a subset of the production data, keeping the sensitive data they contain safe is a big concern. 26% of respondents limit sensitive data to specific users, the most common method here being methods like Role Based Access Control. A further 21% mask or de-identify sensitive data, and 13% replace it with synthetic data.

Worryingly, however, 35% have no approach for protecting sensitive data and 6% are unsure. In simpler terms, four out of ten development teams aren't protecting sensitive data.

Blog: 10 tips for Test Data Management Success

What best describes your approach to handling sensitive data for development and testing?

  • No approach for sensitive data (35%)
  • Sensitive data limited to specific users (26%)
  • Sensitive data masked / de-identified (21%)
  • Replaced by synthetic data (13%)
  • Unsure (6%)

35% of organizations have no approach for protecting sensitive data

Data security and access controls

There are a variety of measures to ensure data security and control access to sensitive data in development as well as production environments. We can see, however, that for every respondent the focus is on production environments. When it comes to testing and development, there is less reliance on security and access controls, despite those environments being more at risk from breaches.

Perhaps surprisingly, data masking and anonymization is also used in only a third (35%) of development and testing environments, with test data management only slightly ahead at 37%. We would have expected this to be higher given the rise in the adoption of Database DevOps which makes both practices far easier to introduce and automate.

One further highlight to mention here is that only 50% of organizations have measures in place for their production environments to be in compliance with industry standards and regulations - and this falls to 37% in development and testing environments.

Learn strategies to safeguard your database estate and minimize reputational risk

What measures has your organization implemented to ensure data security and access controls?

Role-Based Access Control (RBAC)
  • Development & Testing (53%)
  • Production (66%)
Encryption (at rest and in transit)
  • Development & Testing (41%)
  • Production (53%)
Network security
  • Development & Testing (58%)
  • Production (66%)
Auditing and logging
  • Development & Testing (42%)
  • Production (60%)
Data backup and disaster
  • Development & Testing (48%)
  • Production (68%)
Data masking and anonymization
  • Development & Testing (35%)
  • Production (36%)
Regular security assessment and penetration testing
  • Development & Testing (34%)
  • Production (51%)
Compliance with industry standards and regulations
  • Development & Testing (37%)
  • Production (50%)
Test data management
  • Development & Testing (37%)
  • Production (31%)

Only 50%of organizations have measures in place for their production environments to be in compliance with industry standards and regulations

Data security responsibilities

The lack of established processes for provisioning data, variable approaches to handling sensitive data, and less stringent data security measures in development and test environments is a concern. It may, however, be down to the team driving security initiatives, which also appears to be varied, with no common practice across business sectors.

The responses to this question are revealing - and worrying - at the same time. 44% of respondents stated that data security initiatives are shared across multiple roles or the IT department as a whole. While this does invite opinion from a wide range of stakeholders, it also makes it less clear who is ultimately responsible.

Only 18% allocate responsibility for data security initiatives to an IT security team, 11% to a Chief Information Security Officer or CIO, 4% to a Data Privacy Officer or 3% to a compliance team.

We know data security in those test and development environments is difficult at the best of times, with misalignment in terms of where the responsibility sits. That's then coupled with the increasing demand to deliver quality software updates, quickly. This often leads to uncomfortable compromises around either the quality of test data or putting customer information at risk in those lower environments.Saskia Parks

Saskia Parks

Product Marketing Manager, Redgate

Who drives your data security initiatives?

  • Shared across multiple roles (30%)
  • IT Security Team (18%)
  • IT department as a whole (14%)
  • Chief Information Security Officer (CISO)//(CIO) (11%)
  • Management/leadership (8%)
  • Unsure (6%)
  • Cross-functional/department team (5%)
  • Data Privacy Officer (DPO) (4%)
  • Compliance team (3%)
  • Dedicated committee/working group (1%)
  • External Consultants/Vendors (1%)