Today, we launched SQL Clone v2, which removes the previous 2TB size limit and now supports cloning databases up to 64TB. In this article, I’ll explain why I think this is big news for many organizations, and describe briefly the changes to SQL Clone v2 that made it possible.
In larger organizations, such as in the healthcare and financial sectors, databases commonly grow as large as 3-7TB. In such cases, database provisioning can be especially painful, due to the time, disk space and network bandwidth required.
Let’s say you’d like to create five copies of a 5TB database, to provision your development and test environments. You could try copying the database files and mounting them, or restoring a backup onto each of the development and test servers, but that’s going to take many hours, about 25TB of disk space, and a hefty amount of sustained network bandwidth.
SQL Clone takes only one full copy of a database, and from this single ‘image’, creates many ‘lightweight copies’, or clones. Each clone is a normal, fully-working database, but requires only megabytes of disk space, and takes only a few seconds to create.
SQL Clone creates the image, as explained in much more detail by our SQL Clone developer, Chris Hurley, by creating a Virtual Hard Disk (VHD), and then copying onto it the database’s MDF and LDF files. For each clone created from that image, SQL Clone creates a local VHD with a connection back to the image.
In other words, each clone created from an image can access all those same image bytes. All that’s stored locally are data pages containing changes made directly to the local clone database.
Suddenly, you only need one full copy, or image, of that 5TB database, and then from it you can create five clones, or as many as you need, in seconds! Each clone should only need about 100MB of local space, and require minimal extra bandwidth.
However, the blocker for some organizations has been that SQL Clone v1 was built on the version of the VHD format (.vhd) used in older, though still prevalent, operating systems such as Windows Server 2008 R2 and Windows 7. The .vhd format can only support disk sizes up to 2TB, which meant that was the biggest database from which you could create an image.
In SQL Clone version 2 we’ve upgraded the technology to work with VHDX (.vhdx), the Hyper-V virtual hard disk format used in Windows 8 and in Windows Server 2012, and upwards, which support disk sizes up to 64TB. SQL Clone will automatically detect if Windows 8 or Windows Server 2012 or later is installed on the host machine. If so, it will enable users to create clones of databases up to 64TB. If not, it will default to the standard VHD technology with its limit of 2TB.
Redgate developed SQL Clone because we saw an urgent need for a new approach to database provisioning, one that could provide the real databases that teams needed for effective development and testing, but far faster, and more securely than was currently possible.
It seems to have struck a chord. We’ve had great feedback from customers who have reported significant decreases in provisioning times, as well disk space and network bandwidth requirements.
It’s helped development teams migrate to use of isolated development databases, as well as to test code quickly, and more accurately, against realistic data. It’s removed many DBA frustrations with provisioning, such as dealing with multiple provisioning tickets, juggling disk space, purchasing and managing extra storage, and firefighting issues in production post-deployment due to poor testing.
We’re excited to launch SQL Clone v2, to extend these capabilities to databases up to 64TB in size, which even at current rates of data growth should be enough capacity for the foreseeable future!
Want to know what additional features are coming soon? Here’s the SQL Clone roadmap.
Also in DevOps
Working in dedicated development environments for the database is the ideal for many. This is the message we frequently hear throughout the industry from thought leaders, at conferences, and in many w...
Also in Database development
If developers and testers work with a database that has similar data characteristics and is of a similar size to production, it will lead to fewer surprises when deploying database changes and help th...
Also in Blog
These are tough times for businesses in the EU. Is the UK going to be in or out? Will it be a managed deal or no deal? Will it happen next week, next month, or never? What’s the latest twist in the ...
Also about database provisioning
Developers, when working on databases rather than the application code, often find they have less freedom to experiment than they are used to. It's not necessarily easy to set up a database for testin...
Also about SQL Clone
SQL Provision offers users an easier way to manage, organize and make available masked copies of databases
Over the past year Redgate has released some major improvements to SQL Provision, including the ability to modify images with Data Masker and SQL scripts, and create templates for clones, allowing use...