Today, we launched SQL Clone v2, which removes the previous 2TB size limit and now supports cloning databases up to 64TB. In this article, I’ll explain why I think this is big news for many organizations, and describe briefly the changes to SQL Clone v2 that made it possible.
In larger organizations, such as in the healthcare and financial sectors, databases commonly grow as large as 3-7TB. In such cases, database provisioning can be especially painful, due to the time, disk space and network bandwidth required.
Let’s say you’d like to create five copies of a 5TB database, to provision your development and test environments. You could try copying the database files and mounting them, or restoring a backup onto each of the development and test servers, but that’s going to take many hours, about 25TB of disk space, and a hefty amount of sustained network bandwidth.
SQL Clone takes only one full copy of a database, and from this single ‘image’, creates many ‘lightweight copies’, or clones. Each clone is a normal, fully-working database, but requires only megabytes of disk space, and takes only a few seconds to create.
SQL Clone creates the image, as explained in much more detail by our SQL Clone developer, Chris Hurley, by creating a Virtual Hard Disk (VHD), and then copying onto it the database’s MDF and LDF files. For each clone created from that image, SQL Clone creates a local VHD with a connection back to the image.
In other words, each clone created from an image can access all those same image bytes. All that’s stored locally are data pages containing changes made directly to the local clone database.
Suddenly, you only need one full copy, or image, of that 5TB database, and then from it you can create five clones, or as many as you need, in seconds! Each clone should only need about 100MB of local space, and require minimal extra bandwidth.
However, the blocker for some organizations has been that SQL Clone v1 was built on the version of the VHD format (.vhd) used in older, though still prevalent, operating systems such as Windows Server 2008 R2 and Windows 7. The .vhd format can only support disk sizes up to 2TB, which meant that was the biggest database from which you could create an image.
In SQL Clone version 2 we’ve upgraded the technology to work with VHDX (.vhdx), the Hyper-V virtual hard disk format used in Windows 8 and in Windows Server 2012, and upwards, which support disk sizes up to 64TB. SQL Clone will automatically detect if Windows 8 or Windows Server 2012 or later is installed on the host machine. If so, it will enable users to create clones of databases up to 64TB. If not, it will default to the standard VHD technology with its limit of 2TB.
Redgate developed SQL Clone because we saw an urgent need for a new approach to database provisioning, one that could provide the real databases that teams needed for effective development and testing, but far faster, and more securely than was currently possible.
It seems to have struck a chord. We’ve had great feedback from customers who have reported significant decreases in provisioning times, as well disk space and network bandwidth requirements.
It’s helped development teams migrate to use of isolated development databases, as well as to test code quickly, and more accurately, against realistic data. It’s removed many DBA frustrations with provisioning, such as dealing with multiple provisioning tickets, juggling disk space, purchasing and managing extra storage, and firefighting issues in production post-deployment due to poor testing.
We’re excited to launch SQL Clone v2, to extend these capabilities to databases up to 64TB in size, which even at current rates of data growth should be enough capacity for the foreseeable future!
Want to know what additional features are coming soon? Here’s the SQL Clone roadmap.
Also in Blog
I’ve read through a number of the industry thought leaders to get an understanding of how DevOps is being communicated out there. As with so much else in life, you can start at Wikipedia to get a ge...
Also in Database DevOps & DLM
In this three-part series, guest bloggers from DevOpsGuys look at the real role of Ops in DevOps. Where it changes, how it changes, and why Ops has an important part to play in the brave new world of ...
Also in Redgate products
My previous article in this series explained why it's important for a development team to adopt a common standard for formatting SQL code. It also gave a broad overview of the styles and actions withi...
Also about database provisioning
If you have a very large database, up to 2TB in size, SQL Clone will let you copy, or 'clone', that database many times, very quickly, making the full database available in multiple SQL Server instanc...