Today, we launched SQL Clone v2, which removes the previous 2TB size limit and now supports cloning databases up to 64TB. In this article, I’ll explain why I think this is big news for many organizations, and describe briefly the changes to SQL Clone v2 that made it possible.
In larger organizations, such as in the healthcare and financial sectors, databases commonly grow as large as 3-7TB. In such cases, database provisioning can be especially painful, due to the time, disk space and network bandwidth required.
Let’s say you’d like to create five copies of a 5TB database, to provision your development and test environments. You could try copying the database files and mounting them, or restoring a backup onto each of the development and test servers, but that’s going to take many hours, about 25TB of disk space, and a hefty amount of sustained network bandwidth.
SQL Clone takes only one full copy of a database, and from this single ‘image’, creates many ‘lightweight copies’, or clones. Each clone is a normal, fully-working database, but requires only megabytes of disk space, and takes only a few seconds to create.
SQL Clone creates the image, as explained in much more detail by our SQL Clone developer, Chris Hurley, by creating a Virtual Hard Disk (VHD), and then copying onto it the database’s MDF and LDF files. For each clone created from that image, SQL Clone creates a local VHD with a connection back to the image.
In other words, each clone created from an image can access all those same image bytes. All that’s stored locally are data pages containing changes made directly to the local clone database.
Suddenly, you only need one full copy, or image, of that 5TB database, and then from it you can create five clones, or as many as you need, in seconds! Each clone should only need about 100MB of local space, and require minimal extra bandwidth.
However, the blocker for some organizations has been that SQL Clone v1 was built on the version of the VHD format (.vhd) used in older, though still prevalent, operating systems such as Windows Server 2008 R2 and Windows 7. The .vhd format can only support disk sizes up to 2TB, which meant that was the biggest database from which you could create an image.
In SQL Clone version 2 we’ve upgraded the technology to work with VHDX (.vhdx), the Hyper-V virtual hard disk format used in Windows 8 and in Windows Server 2012, and upwards, which support disk sizes up to 64TB. SQL Clone will automatically detect if Windows 8 or Windows Server 2012 or later is installed on the host machine. If so, it will enable users to create clones of databases up to 64TB. If not, it will default to the standard VHD technology with its limit of 2TB.
Redgate developed SQL Clone because we saw an urgent need for a new approach to database provisioning, one that could provide the real databases that teams needed for effective development and testing, but far faster, and more securely than was currently possible.
It seems to have struck a chord. We’ve had great feedback from customers who have reported significant decreases in provisioning times, as well disk space and network bandwidth requirements.
It’s helped development teams migrate to use of isolated development databases, as well as to test code quickly, and more accurately, against realistic data. It’s removed many DBA frustrations with provisioning, such as dealing with multiple provisioning tickets, juggling disk space, purchasing and managing extra storage, and firefighting issues in production post-deployment due to poor testing.
We’re excited to launch SQL Clone v2, to extend these capabilities to databases up to 64TB in size, which even at current rates of data growth should be enough capacity for the foreseeable future!
Want to know what additional features are coming soon? Here’s the SQL Clone roadmap.
Also in Blog
Whether you’re exploring the advantages of DevOps or already fully immersed in the journey, including the database brings additional advantages. But how are you performing compared to the competitio...
Also in Database development
SQL Server 2017 officially landed today and is now on general release. The latest version of the heavyweight platform is more than the sum of its parts, however, because it doesn’t just deliver new ...
Also in DevOps
In February 2001, 17 people met at the Snowbird ski resort in Utah. They were the leading exponents of Extreme Programming, Scrum, and Adaptive Software Development, and they were seeking a set of com...
Also about database provisioning
A common thread behind the software we develop is a problem or issue we've come across in the software development process. We then adopt the software ourselves, which gives us a real understanding of...
Also about SQL Clone
It's a hard and time-consuming task for a DBA to provide developers with development and test copies of a 'production' database. Nevertheless, both the development and operations team have a shared in...