"Disk space problems were disrupting work when developers wanted a fresh database copy."
Even before Paymentsense adopted SQL Clone, the IT team had an efficient database development process. Every developer could develop and test against a copy of the database in their own sandbox, and in the test environment there were often 15 or more copies of the same database being used.
This allowed different branches to be developed at the same time and encouraged and fostered a continuous deployment process, where changes were made in development, tested, and then sent to pre-production before finally being deployed.
The main issues were with the database provisioning process – specifically, the need to keep all development servers, and the 15 databases in the test environment, constantly updated. The time involved in performing the database restore operations and the disk space required were a constant concern for everyone involved.
For the largest databases in the test environment, averaging 200–300GB, in size, Ahmed resorted to copying over the schema only, and then importing a small amount of sample data for testing.
This put a constraint in the testing process because, while the schema was accurate, the data was a representation rather than a true copy. Occasionally, this meant newly-deployed code would throw up unexpected results in production, which they couldn't reproduce on the sample data in the test environment.
"The developers couldn’t rely on test results to reflect accurately the behavior in production."
"We’ve cut the time for database provisioning by more than 85%, which is a really quick win for us."
The development process at Paymentsense has remained broadly the same as it was before SQL Clone.
Introducing SQL Clone has made a big difference. Using its PowerShell interface, an automated process is now in place that runs a backup of the databases during the night, uses SQL Clone to create a data image from that backup, and then stores it on a shared disk. A second process then creates multiple clones from that data image so that they are available the next day, and removes any unused clones and images.
When developers create a new branch and want to test it against the database, the continuous deployment process creates a new clone on demand, so they always have an up-to-date copy to work from. Importantly, with SQL Clone the provisioning takes just seconds to happen. As Ahmed says:
At the moment, they perform data masking as a separate step, by running scripts against each clone. Their next task is to incorporate this step into their automated overnight job. While saving hours of effort, the process has also removed the disk space problem, with each clone taking up only around 40MB, whatever the size of the original database. More importantly, development work is now far more accurate because the clones access a realistic copy of the data in production.