Virtualizing Exchange: points for discussion

With the increasing acceptance of the use of Virtualization as a means of providing server infrastructure, this technology is being applied to production Exchange servers. This is a  solution that is not just limited to the small shop, Nathan Winters discusses the pros, cons and challenges that lay ahead in providing a flexible and highly available email system

 

Introduction

The subject of virtualization has been a pretty hot topic for the last couple of years. It seems to offer a massive amount, for example: consolidating underused servers and therefore providing cost savings on hardware, rack space, electricity and cooling. On top of that, virtualizing your server infrastructure brings other benefits such as the ability to easily move a virtual machine from one piece of physical hardware to another and also to rapidly provision new servers where required.

Although this sounds great, there has been somewhat of a problem  with virtualising  Exchange 2007  because,  Microsoft has not, until very recently, supported Exchange 2007 on any virtualization platform. To be fair though, that has not stopped a lot of people from putting their Exchange servers on a virtual platform. In fact it is something I have, myself, done for a handful of clients utilising the VMware ESX platform. However, as mentioned, the support situation has now changed, and these changes form the topic of the next section.

Is it supported?

Microsoft has not,
until very recently,
supported
Exchange 2007
on any virtualization
platform

In this rapidly changing market place, there is no straight-forward answer to the question of support. Essentially, as far as Microsoft is concerned, this all boils down to what they have tested. Exchange 2003 has long been supported on the Microsoft Virtual Server platform, although only if you had a Microsoft Premier Support Contract. I have seen almost nobody using Virtual Server as anything other than a test platform, and other products like VMware Workstation were rather better for that. So why is Exchange 2007 different? It is because Exchange 2007 is the first Microsoft Server application which required a 64 bit (x64) operating system (OS). At the release of Exchange 2007, Microsoft still only had Virtual Server as their virtualization offering, which does not support the 64 bit OS required to run Exchange 2007. Therefore, Exchange 2007 was not supported in a virtual environment as Microsoft would have been reluctant to test on another company’s virtualization platform!

Since July 2008, when Microsoft released Hyper-V, Microsoft’s new hypervisor based virtualization platform, it now has a virtualization platform capable of supporting 64 bit operating systems. Therefore it came as no surprise that Microsoft issued a new support statement in August this year which is found at the link below:

Microsoft Support Policies and Recommendations for Exchange Servers in Hardware Virtualization Environments

Some of the key points are:

  • The virtualization software must be one or other flavour of Hyper-V, or if 3rd party, must be listed on the Windows Server Virtualization Validation Program (SVVP).
  • Exchange must be running on Windows Server 2008

     

  • The Unified Messaging role is not supported, although all others are.
  • Virtual disks that dynamically expand are not supported by Exchange.
  • Virtual disks that use differencing or delta mechanisms (such as Hyper-V’s differencing VHDs or snapshots) are not supported

     

  • Snapshotting Exchange server virtual machines is not supported, as this is not an Exchange aware backup mechanism

     

  • Both CCR and SCC are supported in hardware virtualization environments provided that the virtualization environment does not employ clustered virtualization servers. .

The final issue listed here deserves some clarification as it has already caused some confusion. Essentially you can do one or the other of the following where the preference would be the second option: (1) cluster the hypervisor roots, or (2) cluster the guests (using multiple roots, for example CCR where one node is a VM on one root and one node is a VM on another root).  You cannot, however, combine the two, which suggests that using technology like VMware’s VMotion and HA/DRS is not supported in conjunction with CCR or SCC. Perhaps this again will change when Microsoft release they Live Migration technology in R2 of Hyper-V although having said that, Microsoft currently has Quick Migration which also isn’t supported when using Exchange clusters. We will simply have to wait and see.

Although the use of hypervisor high-availability techniques, in conjunction with Exchange clusters, is not actually supported, using VMware VMotion to move the active node in a CCR cluster does actually work (on a test system at least), and doesn’t seem to cause a failover either! You do get a bunch of Event ID 1122 and 1123 messages telling you about the lost network connectivity, but things appear to keep working. Of course this may well not be true for a heavily loaded system as depending on the amount of time required to VMotion, a failover may be triggered. All in all, it isn’t supported and frankly, going to the trouble of setting up an high availability system, only to run it in an unsupported way, seems rather perverse to me!

Alongside the support announcement discussed above, Microsoft also changed some licensing conditions as the below quote from the Exchange team blog describes:

Microsoft is waiving its 90-day license reassignment policy to enable customers who virtualize Exchange to move their licenses between servers within a data farm as often as necessary.

So all in all this means that it is very much supported to run Exchange under the conditions stated on the Microsoft Hyper-V platform and to have flexibility to move virtual machines between physical hosts. What I guess a number of you are wondering is what does this mean for VMware support? Well to answer that we have to take a look at the Server Virtualization Validation Program or SVVP.

The website for SVVP can be found at the link below:

Server Virtualization Validation Program

Essentially this is a way for Microsoft to certify whether 3rd party virtualization products can adequately serve Microsoft Windows and the Microsoft software which runs on Windows. VMware is on this list as supported however, what is critical to note with regards Exchange is the supported processor architecture type. The link below lists the platforms which have met the requirements of the program for x64 processors:

Server Virtualization

Interestingly this list changed during the writing of this article (Oct 2008). When I started the article, VMware was not on the list! However, during the first revision process, VMware became a supported x64 platform with the only caveat being that virtual machines can only have 16GB of RAM allocated to them.

I hope having read the above you have an understanding of the issues surrounding the support of Exchange on a virtualization platform. Before we move on, I would simply like to add that I and many other people have been running Exchange 2007 on various virtual platforms and it does generally work very well. The support issue is a risk, but often it is not a big enough risk to stop people making use of the benefits or virtualization technology. Having addressed the support issue let’s now move on to take a look at virtualizing Exchange in practice.

Virtualizing Exchange in practice

In this section I will look at the pros and cons of virtualizing Exchange. I must make it clear at this point that I have not yet had the opportunity to deploy Exchange 2007 on Hyper-V, although I have deployed Exchange 2007 on VMware ESX. Therefore, my comments on Hyper-V are based on numerous discussions with trusted colleagues who are specialists in the Virtualization field.

Benefits of Exchange Virtualization

Exchange is a massive
 product and getting
familiar with it all is
 not easy,

As you will know if you run an Exchange organisation, Exchange is not a simple application. Exchange is a massive product and getting familiar with it all is not easy, therefore, the ability to have a replica of your production environment is extremely helpful when it comes to the testing and validation of upgrade and migration work. Virtualization makes this extremely simple as copies of existing physical, and virtual machines, can be taken, and then run in isolation from the main network.

The introduction of multiple roles in Exchange 2007 has been very helpful in allowing Exchange to scale well. However, it has also pushed up the server count dramatically. Virtualizing the Exchange environment allows these multiple roles to still run on separate virtual machines which can be tuned accordingly whilst keeping down the number of physical machines required.

Following on from the flexibility of providing a test lab, it is common nowadays, that this lab solution be provided on equipment available for disaster recovery. The ability to move a Virtual Machine from one piece of hardware to another means that in a disaster, getting the Exchange services up rapidly is much simpler than when relying on the correct physical hardware being available.

Interestingly, and one area which I did not expect to put in the benefits section is performance. It would appear that when sized according to Microsoft Exchange sizing guidelines, a virtualized Exchange infrastructure can perform almost as well as a physical one. This was discussed at VMworld this year. In particular, it would seem that message throughput is actually slightly better on a virtual Hyper-V platform than on physical hardware.

Another area which could be considered a benefit and a possible problem, is management. The reason I mention it as a possible problem, is simply that it is another layer of management technology, however, once you accept the necessity, then management options are a definite benefit. It is one area in which Microsoft excel. Although VMware have their Virtual Center console which is not a bad solution, the Microsoft solution, System Center Virtual Machine Manager (VMM), to give it its full name, gives you cradle to grave, hardware to application management.  VMM 2008, like Virtual Center, has to be purchased however, what is brilliant is the way it integrates with System Center Operations Manager 2007 using a connector/management pack.  That gives you expertise on your infrastructure that’s built into the network.  Want to know which servers have spare resources to be potential hosts?  Want to know which machines should be converted to VM’s (and then P2V them).  Want performance/health information in a single integrated management infrastructure (System Center)?  This complete management solution is something the competition struggles to match. 

 Problems with Exchange Virtualization

Of course there are some problems with virtualizing Exchange. It is important not to think that virtualization gives endless resources. It is absolutely critical to size the servers just like you would before. On top of this, running a virtual platform gives an added layer of complexity that must be understood and managed carefully so as to provide a good platform for the virtual machines it supports.

 Having mentioned performance in the benefits section, I think it is worth entering it here too. Why? Because I still feel that putting another layer underneath something that is already IO and memory intensive isn’t necessarily the greatest idea. After all, if you only run one VM on the box why not just use physical hardware? I feel that this is particularly true when running the Exchange Mailbox server role on a virtualization platform. Whilst it is true that when using pass through disks performance is often within 1% of the physical hardware it is however, still true that when load in particular on the network cards increases performance problems can occur. There will be improvements in this area soon, as new network cards increase throughput by implement virtual switches in hardware. Whatever, it is not recommended to virtualise more than one Mailbox server on a single virtual host machine.

Still there are benefits to virtualization so at this point it is perhaps a question of whether these outweigh the fact you may get less users on a VM than when using a physical machine. Looking at non mailbox roles, Unified Messaging is simply not supported and really doesn’t scale well even if you try to put it in a virtual environment, as the audio playback can become rather choppy! A possible barrier to virtualizing the Client Access and Hub Transport roles is the implementation of Windows Network Load Balancing (WNLB) on the virtualization platform. This is something that I have struggled with however; it would seem that it is possible but that problems can occur unless things are configured correctly as described at the links below. To be fair this is no different when in a physical environment.

Exchange 2007 Unicast NLB Issue on Hyper-V

Although virtualization brings some benefits for disaster recovery as mentioned above, one can’t get away from the fact that by virtualizing you are putting a number of servers on a single physical piece of hardware. Obviously there is a need to mitigate this single point of failure and one method is to use redundant hardware including PSUs and NICs. This is one area to be particularly careful of on the Hyper-V platform, as under Hyper-V physical NIC teaming is not currently supported, so if you lose a NIC, everything on that machine loses connectivity!

Summary

Microsoft Hyper-V, VMware and other virtualization platforms provide a great platform for Exchange especially as you now have the comfort of knowing that the solution is supported. Realistically it is very likely that as virtualization becomes more and more accepted as the normal way of providing server infrastructure, production Exchange servers  will be virtualized.

In my opinion, there are clearly a few challenges to be faced when virtualizing Exchange, but so long as the challenges/limitations of the infrastructure are understood, the guidelines laid out by manufacturers are followed and you thoroughly test the performance of the implementation before rolling it out in production, virtualizing Exchange can be very successful.

What is very clear is that virtualizing Exchange is no longer just an option for the small shop, but is now a solution for even the largest Exchange deployments, looking to provide a flexible and highly available platform for their email system.