A Beginner’s Guide to Virtualizing Exchange Server, Part 2

It isn't easy to measure the consumption of physical resources by servers in virtual machines, since each partition has its' own virtualised view of system resources. Not only that, but there is a subtle difference between virtual processors and physical processors. Brien Posey explains the special performance counters that can be used to get an accurate assessment, and goes on to describe how to test your physical servers to see if they are good candidates for virtualization.

If you read my first article in this series, then you got something of a crash course in Hyper-V’s architecture. Of everything that I covered in that article though, there are two main points that you need to keep in mind:

  1. When it comes to virtualizing Exchange Server, it is critical that you monitor resource consumption so that you can ensure that there are sufficient hardware resources available to effectively service Exchange and any other virtual machines that may be running on the server.
  2. Each of the various resource monitoring mechanisms that I showed you tells a completely different story regarding how much of the server’s physical resources are actually being consumed.

In other words, it is important to find out how much of the server’s resources are being consumed, but you are not going to be able to do so in the usual way.

So Why the Discrepancy?

In my previous article, I showed you that when you monitor the amount of CPU time that the server is consuming, you will get different results depending upon where the measurement was taken. I never really said why though. The reason for this is that neither the root nor the child partitions control the APIC timer. In essence, this means that each partition has its own virtualized view of the system’s available resources. It isn’t that one set of performance counters is right and another set is wrong. All of the counters are correct from their own frame of reference.  Remember that the goal of having multiple partitions is to isolate the virtual machines from one another. Therefore, no one single partition (including the root partition) has the full picture of how the system’s resources are being used.

Monitoring CPU Utilization

Traditionally, the \Processor(*)\% Processor Time performance monitor counter has been used to find out how much CPU time a server is consuming. As you have seen though, this counter does not tell you the whole story when you use it in a Hyper-V environment.

There are ways of measuring CPU utilization in a Hyper-V environment, but before I show you how, it is important that you understand the difference between virtual processors and logical processors. Logical processors map directly to the number of CPU cores that are found in the system. For example, if you have a server that has two quad core CPUs installed, then the server has a total of eight CPU cores. Since logical processors correlate directly to the number of CPU cores, the server also has eight logical processors.

Hyper-V partitions do not directly use logical processors. Instead, they use virtual processors. As I explained in the first part of this series, you can actually allocate more virtual processors to your partitions than the machine has logical processors. Therefore, if you want to know how much of the server’s CPU resources are actually being consumed then you need to look specifically at the virtual processor utilization.

If you have ever used the Performance Monitor within the host operating system, then you may have noticed that there are a number of Performance Monitor counters that are specifically related to Hyper-V. One of the most useful of these counters is the \Hyper-V Hypervisor Logical Processor (_Total)\% Total Run Time counter. This counter shows you the cumulative CPU consumption across all virtual processors. The output from this counter is presented as a percentage of the server’s total CPU resources, just as the \Processor(*)\% Processor Time counter does on a non virtualized system.

Microsoft even gives us some guidelines as to what is considered to be an acceptable level of utilization. According to Microsoft, if this counter is reflecting a value of 60% or less then the server’s CPU utilization is healthy. If the counter reflects a value of 60% to 89% then the server needs to be monitored, because it is in danger of exhausting available CPU resources. A value of 90% or above is considered to be a critical state in which the server’s performance will suffer.

That is how you measure the server’s CPU consumption as a whole. There are a couple of other Performance Monitor counters that you need to know about though; the \Hyper-V Hypervisor Logical Processor( _Total)\% Total Run Time and \Hyper-V Hypervisor Virtual Processor( _Total)\% Total Run Time counters.

The \Hyper-V Hypervisor Logical Processor( _Total)\% Total Run Time counter shows you how hard the CPU is working from a logical processor standpoint, while the \Hyper-V Hypervisor Virtual Processor( _Total)\% Total Run Time counter looks at the workload from a virtual processor standpoint. Generally speaking, these two counters should be balanced so that they display roughly about the same value.

 If you find that logical CPU utilization is high, but that virtual CPU utilization is low, then it is often a good indication that there are more virtual processors allocated to the server’s partitions than the machine has logical processors. Although you can create more virtual CPUs than the number of logical CPUs that the server contains, you should try to use a one to one ratio when possible.

If you find that you have allocated an excessive number of virtual processors then Microsoft recommends using the \Hyper-V Hypervisor Virtual Processor(*)\%Guest Run Time counter to determine which virtual processors are consuming the most CPU resources. You can then begin removing lesser used virtual processors from guest machines in an effort to try to reach a one to one virtual to logical processor ratio.

On the flip side, you may occasionally run into situations in which the logical processor utilization is low, but the virtual processor utilization is high. When this happens, you should consider allocating additional virtual processors to the partitions that need them most. Of course you have to make sure that the operating system that is running within that partition will support the extra processors before you actually begin allocating any additional virtual processors. If you find that you have CPU resources to spare, but the guest operating systems do not support adding additional processors, then you might consider adding additional virtual machines to the physical server to make better use of the server’s resources.

Planning for Exchange

The techniques that I have shown you work well for figuring out how much of a server’s resources are being consumed, but they may not do you a lot of good if you are trying to figure out whether or not a server has the resources to host a virtualized Exchange Server. Fortunately, there are a couple of tricks that you can use to gauge a server’s capabilities before you try to virtualize your Exchange Server.

The first thing that you need to keep in mind is that from a hardware requirement standpoint, Microsoft doesn’t make any differentiation between a physical server and a virtual server. For instance, suppose that the Exchange Server role that you plan to deploy requires 2 GB of disk space. That 2 GB requirement remains the same whether you are going to be deploying Exchange on a physical server or on a virtual server.

Of course adhering to the hardware requirements for Exchange Server will only get you so far. You have probably noticed that I have spent a whole lot of time in this series discussing how you can figure out how much of a server’s CPU time is available. The reason for that is that Hyper-V makes it really easy to create a virtual server that adheres to specific hardware specifications. For instance if you need a virtual machine with 2 GB of RAM, three hard drives that are 100 GB each, and two network adapters, then you can easily create such a machine by using the Settings option in the Hyper-V Manager.

CPU resources are a little bit more difficult to allocate though. It’s easy to tell Hyper-V how many virtual processors you want to assign to a virtual machine, and you even have the option of allocating CPU resources as a percentage of the server’s total CPU resources.  What you can’t do however, is to tell Hyper-V to give your virtual machine the equivalent of a 2.8 GHz quad core processor. In other words, you can tell Hyper-V how many virtual processors to assign to a virtual machine, or you can tell it to use a percentage of the overall CPU resources, but there isn’t a way of requesting a specific amount of processing power. That’s why CPU monitoring and capacity planning is so important.

Of course this still leaves the question of how you can determine whether or not your host server is up to the job of running a virtualized instance of Exchange Server. If your Exchange Servers are currently running on physical hardware then the assessment process is easier than you might think.

The Microsoft Assessment and Planning Toolkit

Microsoft offers a free utility called the Microsoft Assessment and Planning Toolkit . This utility can be used for a lot of different purposes, but one of the things that you can use it for is to test your physical servers to see if they are good candidates for virtualization.

Unfortunately, this utility isn’t Exchange Server aware, so it isn’t going to tell you that you can’t virtualize one of your Exchange Servers because it isn’t running a configuration that is supported in a virtual environment. What it will do, is collect performance data from your physical servers and then use that performance data to determine whether or not the server is a good candidate for virtualization based on its resource consumption, and on the resources that are provided by your host server.

I recommend running the Assessment and Planning Toolkit on a computer that’s running Windows Vista, rather than running it directly on one of your servers. That way, you can minimize the utility’s performance impact on the servers that you are going to be benchmarking. However, the machine that you run the utility on needs to have Microsoft Office installed. You won’t even be able to install the utility unless you install Microsoft Office first.

Gathering Performance Data

The first step in analyzing your physical servers is to create a simple text file containing the NetBIOS names of the physical servers that you want to analyze. Just use Notepad to create the list, and put each server name on its own line.

Now that you have created a list of servers to analyze, open the Microsoft Assessment and Planning Solution Accelerator. When the console opens, click on the Select a Database link. You should now see a dialog box asking you if you would like to select an existing database, or if you would like to create a new database. Since this is the first time that we have used the Assessment and Planning Toolkit, choose the option to create a new database. Provide the database with a name, and click OK.

Next, select your newly created database from the console’s Assessment pane, and then click the Prepare Recommendations for Server Consolidation Using Windows Server 2008 Hyper-V or Virtual Server 2005 R2. You will now see a message explaining that the wizard that you have chosen to use requires performance data. The message contains a link that you can use to capture performance data for your computers. Go ahead and click this link.

At this point, you will be prompted to supply the name and path of the text file that you created earlier. After doing so, click Next and you will be prompted to provide a set of WMI credentials for the computers that you have specified. You can use the same credentials for each computer on the list, or you can specify separate credentials for each machine if necessary. When you are done, click Next.

You should now see a screen that asks you when the benchmark tests should complete. By default the tests are set to run for an hour, but you can specify a different period of time if you wish. Keep in mind though, that if you set the testing period to be too short, then the wizard may not have enough data to make a good recommendation. Click Next, and you will see a summary of the settings that you have chosen. Take a moment to review these settings, and then click Finish. The performance monitoring process will now begin.

Analyzing the Performance Data

Now that we have completed the data collection process, it is time to analyze the results. To do so, click on the Prepare Recommendations for Server Consolidation Using Windows Server 2008 Hyper-V or Virtual Server 2005 R2 link. When you do, Windows will launch the Server Virtualization and Consolidation Wizard.

The first thing that you will need to do when the wizard starts is to select the virtualization product that you plan on using. The wizard allows you to select either Hyper-V or Virtual Server 2005 R2.

Click Next, and you will be asked to provide some details regarding the host machine’s CPUs. The wizard asks some fairly detailed questions regarding things like the sizes of the level 2 and level 3 caches, and the bus speed. Granted, answering these questions isn’t rocket science, but it may require you to look up your server’s specs. If you don’t have a way of getting the specific details for your server’s CPUs, then you can just select the make and model of the CPUs that the server is using, and the wizard will fill in the details for you. They might not be completely accurate in every situation, but they should at least be close enough to make the wizard’s results reasonably accurate

After you have filled in your CPU information, click Next, and you will be prompted to enter some details about your host server’s disk configuration. This information is fairly easy to fill in. You just need to know the type and speed of the disks that will be used, and what type of array configuration (if any) will be used. When you are done, click Next.

The following screen prompts you to enter the number of network adapters that are installed in the host server, and the speed of those adapters. This screen is also the place where you tell the wizard how much memory is installed in your host server.

Click Next, and you will be asked if you would like to set a limit as to the number of virtual machines that the server can host. Personally, I recommend setting a limit, although the actual limit that you set is going to depend on what you feel comfortable with. The reason why I recommend setting a limit is because the recommendations that the wizard is going to make are based on an hour’s worth of performance monitoring. Your peak usage may cause significantly higher resource consumption than what was recorded during the monitoring period. Besides that though, it is smart to leave some room for future growth.

Click Next, and you will be asked to provide a text file containing a list of computer names. You should use the same text file that you used when you gathered performance data on your servers.

Click Next one more time, and you will see a screen that displays all of the settings that you have entered. Take a moment to verify that you have entered this information correctly, and then click Finish. The Microsoft Assessment and Planning Solution Accelerator will now create the requested reports.

Viewing the Results

When the report generation process eventually completes, click Close, and you will be returned to the main console screen. You can access the reports by clicking the View Saved Reports and Proposals link. When you do, you will find that the wizard has created two documents. The first document is an Excel spreadsheet that contains the raw performance data from each of the servers that you are considering as virtualization candidates. The second document is a Microsoft Word document that contains detailed recommendations for virtualizing your proposed servers. These recommendations are based on the performance data that was collected and the information that you provided about your host server.

One thing that is important to understand is that the wizard does not perform any types of benchmarks on your host server. Therefore, the recommendations that the wizard makes are only as good as the information that you provide it with.

Conclusion

There are obviously many more virtual server resources that you can monitor other than just the CPU. Keep in mind though, that once you understand the basic concepts involved in monitoring CPU resources, many of those same concepts can be applied to monitoring other types of resources.