The Customer

DataPaws is a US-based data consultancy specializing in SQL Server, data engineering and operational reliability. The team works closely with organizations operating complex data estates, helping them modernize critical processes while maintaining stability and compliance.

One of its clients, a large US financial services organization, operates a complex IT estate. The organization runs around 40–50 business-critical applications accessing hundreds of databases across multiple systems, with some individual databases reaching several terabytes in size. The majority of the estate runs on-premises, with a smaller but growing footprint in the cloud with Azure. Managing this environment is a team of more than 30 developers, as well as a dedicated data engineering team, with Ben Wiggin, Founder at DataPaws, as their primary DBA.

The Challenge

Because many of the developers and contractors work exclusively in non-production environments, sensitive production data must be reliably masked when creating test datasets, before refreshes can take place. To accomplish this, the organization relied on a legacy enterprise masking tool that was slow, opaque and risky to operate.

Masking two key databases involved a manual, multi-stage extract–transform–load process. One database alone required more than 20 hours to complete, with each stage needing to be started manually, often overnight and into weekends, with any failures adding to the long hours and precariousness of the situation. “With the old tool, if something failed there were no outputs or logs to explain why. You’d just restore the database and start again, hoping it worked the next time”, said Ben Wiggin, Founder at DataPaws.

Over time, the tooling became effectively untouchable. Configuration was inherited via spreadsheets and a 30-page instruction document, and no one felt confident adding new columns, masking additional data, or making changes without risking failure. Updates were only every six months, and the process depended heavily on a single individual being available at exactly the right time.

The breaking point came when the organization upgraded to SQL Server 2022. The masking tool no longer worked and upgrading it would have required a complete reimplementation, right before a scheduled refresh that was required to align with regulatory requirements.

At that point, the organization had limited time to find a replacement that could deliver the same coverage without introducing additional risk.

The Solution

With roughly 30 days to act, DataPaws implemented Redgate Test Data Manager as a direct replacement for the legacy masking solution. The priority was not to redesign the process, but to reliably reproduce existing masking coverage in a way that was faster, safer, and easier to maintain under operational pressure.

The decision was pragmatic. The team was already familiar with Redgate tooling and documentation, and Redgate Test Data Manager offered a way to meet the compliance deadline without introducing additional risk or complexity. The focus was on choosing a solution that could be implemented quickly and trusted immediately.

Using existing documentation as a baseline, Ben recreated the masking rules in Redgate Test Data Manager and validated them against the same datasets used previously. Out of the box, the product covered the vast majority of masking requirements, with only a small number of edge cases requiring custom handling. Those customizations were straightforward to configure and did not slow down the overall implementation.

A key requirement was preserving referential integrity while masking sensitive data. For personally identifiable information such as addresses and zip codes, the data needed to remain realistic so that non-production environments accurately reflected production behavior. As Ben outlined, “One of the things that Redgate Test Data Manager has definitely helped with is not only masking the data but masking it and keeping it real. It helps us better align to keep our non-prod closer to our prod environment.” Redgate Test Data Manager ensured related values remained consistent across tables and generated realistic masked data, improving the quality and reliability of downstream testing.

The solution also addressed limitations in the legacy tooling around complex and evolving schemas. Redgate Test Data Manager was able to handle encrypted columns and other data types that had previously caused issues, making it easy to identify and mask new tables or columns as the schema changed.  Something the previous process could not support without significant risk. Also, because Redgate Test Data Manager masks data in place, there was no need to extract sensitive information to external servers, removing a major source of both delay and exposure risk in the old approach.

Despite the tight timeline, the implementation was production-ready in under two weeks, with core masking operational in just eight business days. Ben explains that “We had Redgate Test Data Manager configured in less than a day, and in actual time it only took 10 hours to replace the other tool with Redgate Test Data Manager”.

The new process no longer being dependent on a single individual was a huge factor. Clear documentation and predictable workflows meant that anyone on the team could now run the masking process when needed, sharing responsibility and future-proofing the operation as the environment continues to evolve. As Ben highlighted this was a huge risk reduction for management, “Being able to have something documented and now have a process where, if I’m out and we need to mask something, someone else could go in and run the masking. That was the biggest thing from a management perspective”

The Results

The impact of moving to Redgate Test Data Manager was immediate and measurable.

What had previously taken more than 20 hours per database now completes in under 45 minutes for both databases combined. A process that once consumed an entire weekend, required overnight monitoring, and left little margin for error can now be completed predictably.  As Ben described, “That alone gives everyone their weekend back, I don’t have to wake up in the middle of the night to start the next step and hope nothing failed.”

Beyond speed, the organization gained visibility and resilience enabling:

• Clear, auditable masking rules visible directly in the tool
• Meaningful error reporting instead of guesswork
• Documentation reduced from 30 pages to two
• Reduced reliance on a single individual to run the process
• Improved regulatory compliance with accompanying documentation and reporting

As well as giving the team a better work/life balance, developers now have the confidence in the tool to innovate and implement new columns and masking.

From a testing perspective, preserving referential integrity and realistic data patterns has improved the quality of non-production environments. Masked data behaves much more like production data, allowing developers and testers to work with greater confidence that issues identified in non-production environments will reflect real-world behavior. Ben summarized this saying, “With the old tool, the data just looked random, now non-production behaves much more like production, without exposing real data.”

Looking ahead

What started as an urgent replacement has become a foundation for improvement. Faster, safer masking is enabling more frequent refreshes and opening up environments that previously avoided production data entirely.

Ben explained, “It’s night and day compared to what we had before. We didn’t just make it faster; we made it understandable and maintainable.” As a result, the organization is now looking to expand the use of Redgate Test Data Manager to other teams and environments, replacing additional legacy masking workflows and establishing a more consistent approach to test data management across the estate.

Furthermore, there is interest in exploring other Redgate solutions, including Redgate Monitor, to improve visibility and operational confidence across the wider database platform.