OCS Disaster Recovery Part 1

Office Communications Server stores its data in a variety of locations, which can make backing it up and disaster recovery a slightly tricky business. Thankfully, Johan Velduis is here to tell which databases and fileshares we need to worry about, and give us peace of mind.

In my last series of articles, I explained how you can build an OCS environment which you can use for instant messaging, conferencing and some other neat features. However, as with every environment, we will need to create backups of this shiny new environment in case a server crashes, or worse. So, in this new series of articles, I will explain how you can backup your OCS environment, and how you can restore it if needed.

Where is your OCS Data?

To better understand what we actually need to backup, we should first know that OCS stores its data in several different places:

  • Databases
  • File shares
  • Active Directory

Let’s have a look at all three of them separately.

Databases

Let’s start with the databases, of which each OCS environment has the following four:

  • RTC (Real-Time Communications):
    Used for storing persistent user data, such as access control lists, allow/block lists, contacts, home server/pool and scheduled conferences.
  • RTCConfig:
    Used for storing persistent OCS 2007 R2 global-level, pool-level and computer-level settings.
  • RTCDyn:
    Contains user data which is transient, such as endpoints and subscriptions, as well as active conferencing server and transient conferencing states.
  • RTCAB:
    Contains the address book.

The location where the databases are stored is different between OCS Standard and Enterprise Editions. In the case of an OCS Standard Edition installation, data is stored in the locally-installed Express edition of SQL  Server 2005. The databases and logs themselves are created, by default, in the root of your C: drive, in the folders RTC, RTC Data, RTC Dynamic Log and RTC Log. Alternatively, if you have an OCS Enterprise Edition, data will be stored on a separate SQL 2005 or 2008 server. Besides these standard databases, an OCS environment can contain some optional extra databases, which are mandatory  if you are using these additional server roles/applications:

  • ACDDyn:
    Contains transient data for the Response Group Services, i.e. which agents are logged in, call waiting data, etc.
  • LCSLog:
    Used by the Archiving server to store archived data, such as Instant Messages and Conference data.
  • RTCCDR:
    Used by the Monitoring server to store Call Detail Records (CDR).
  • QoEMetrics:
    Also used by the Monitoring server, but stores Quality of Experience (QoE) data.
  • GroupChat:
    Used to store chat history, system settings and metadata specific to Group Chat.
  • GroupchatCompliance:
    Used by the Compliance service to store chat history and compliance data. This data could also be stored in the same database as the GroupChat server uses, above.

OK, these are all the databases which can exist in an OCS 2007 R2 server. So what options do we have to create a backup of them, and do we need to create a backup of all of them? Well, there are several options available for how to create backups:

RTCConfig contains data which can be also backed up and restored another way. Specifically, you may have heard about the LCSCmd.exe command, which can be used for several operations: preparing the Active Directory, setting the external webfarm URL, and yes, also for backing up and restoring the global-level, pool-level and computer-level settings. The RTCDyn database doesn’t have any alternative backup methods, but it just contains transient data, so you may decide to not back this up.

File Shares

You might wonder at this point, “does OCS really use file shares to store data?” Yes it does, so let’s list all of them:

  • Meeting content:
    A Fileshare which contains uploaded content, Q&A logs, polling data and chat data of meetings.
  • Meeting content metadata:
    An XML file containing information about uploaded content, such as the date and time a file is uploaded.
  • Meeting Content compliance:
    Another XML file, which contains records of the upload activities.
  • Address book files:
    The description says it all, I think. The address book files for OCS are also stored in a file share.
  • Application data files:
    Data for applications, such as the response groups and other 3rd party applications, is stored here.
  • Group Chat web and compliance folders:
    These contain files which are uploaded to the Group Chat Web Service.
  • Group Chat compliance XML files:
    These contain compliance data for the Group Chat functionality.
  • Update files:
    These contain the update files for devices.

The location where these files are stored, once again, depends on the version of OCS you are using. To make it a little bit more clear, here’s an overview:

Data

Standard Edition

Enterprise Edition

Meeting content

<drive>:\Program Files\Microsoft Office Communications Server 2007\Web Components\Data MCU Web\Web

User defined UNC path

Meeting content metadata

<drive>:\Program Files\Microsoft Office Communications Server 2007\Web Components\Data MCU Web\Non-Web

Meeting content compliance

User defined UNC path

Address-book files

<drive>:\Program Files\Microsoft Office Communications Server 2007\Web Components\Address Book Files

Application data files

<drive>:\Program Files\Microsoft Office Communications Server 2007\Application Host\Application Data

Group Chat web and compliance folders

User defined UNC path

Group Chat compliance XML files

User defined UNC path

Update files

Client:
<drive>:\Program Files\Microsoft Office Communications Server 2007\Web Components\AutoUpdate

Devices:

<drive>:\Program Files\Microsoft Office Communications Server 2007\Web Components\DeviceUpdateFiles

A backup of almost all the data I’ve mentioned needs to be made. The only exception is the Address-book files, as these will be regenerated automatically.

Active Directory

When implementing OCS, you will need to make some modifications to the Active Directory; specifically, prepare the schema, prepare the forest and prepare the domain. During these operations, a lot of information is added to Active Directory which is then used by OCS.

For example, when activating a server role, the setup process will create a service principal name (SPN) and register it to run as a service. This is just one simple example, but there are many more things which are stored in the Active Directory which are used by OCS, and because of this we will need to create a complete backup of the Active Directory so no data will be lost. Most backup software packages will support creating a backup of the Active Directory. If you don’t have any such software, buy some now, because losing your AD has a lot of drastic consequences, and not only for OCS.

Utilities

OK, enough about the content which we need to backup; let’s have a look at the utilities and software we can use to actually do it. We’ll have a quick look at what’s available (and some considerations to bear in mind), and then we’ll jump straight into a demonstration.

LCSCmd

One of the utilities which is delivered with OCS 2007 R2 is LCSCmd, which, as I’ve briefly mentioned, can be used for several things. Besides querying for settings, modifying settings, preparing the AD, etc., you can also use this handy utility to back up and recover OCS-specific settings. However, there are three exceptions to this: LCSCmd cannot backup the settings for the Group Chat, Group Chat Compliance or Communicator Web Access virtual servers. But how do you use the LCSCmd command? Here are some examples:

The above command will export the global and pool settings of the OCS environment. This is done by specifying the global,pool value for the /level parameter. If you like, you can create a separate backup of the global and pool settings, though in most cases the option above is chosen so one file contains both settings.

The third possible value of the /level parameter is machine, which backs up only server-specific settings. In the case of the Front-End of an OCS Standard Edition, this will also backup the settings of the other services running on it. The question is, which servers should you make a machine-settings-level backup from? Well, the answer is quite simple: every server which has OCS components installed on it, except for the Group Chat, Group Chat Compliance and Communicator Web Access virtual servers. You will find an overview hereof which servers you will need to create a machine-level backup of.

Windows Server Backup/3rd Party backup software

The other utility you will need to have is a software tool which will let you create backups of your databases, file shares, AD and, if you like, system state and other data. For this, you will find multiple solutions delivered by various vendors and, fortunately, one of them is already included in your Windows OS: Windows Server Backup. As well as being free and already installed, with this tool you can create the backups you want without additional agents.

That might not sound like much, but to use most 3rd party products, you will need to buy some extra agents to create backups of open databases. Or, alternatively, use the SQL management tools to create a backup of the database, and then back that up using the 3rd party software. As messy as this might sound, my advice is definitely to follow your own company’s guidelines. So, if a particular backup software package is the company standard, don’t try to save costs by using some another method, but just use an agent of the software currently in use to back up your OCS environment. Trying to create and use your own method around an already established procedure is just a recipe for disaster.

Let’s Start the Demonstration

OK, first let’s look at a short description of my environment. I have a Windows 2008 R2 Domain Controller, and OCS 2007 R2 Standard Edition installed on a separate Windows 2008 R2 server. If you’re wondering whether Windows 2008 R2 is supported, I can thankfully say that, yes, OCS 2007 R2 has been supported on Windows 2008 R2 since March 30th 2010. My OCS server is named ocs2007, and since this is a Standard Edition OCS 2007 R2, this will also be the poolname.

Settings

Now that you know what my environment looks like, let’s start with backing up the global, pool and machine specific settings by using LCSCmd.exe. To back up the settings we will execute the following command:

Obviuosly, you need to make sure the backup directory exists, or else you will get an error when executing the command.

1060-figure1.jpg

Figure 1. Backing up the Global and Pool settings using LCSCmd.exe

As you can see from Figure 1, the backup went successfully, and has correctly created a backup of the settings. This is all done by making a connection via the Windows Management Instrumentation (WMI), and the result is exported to an XML file. Next will be the machine specific settings:

1060-figure2.jpg

Figure 2. Backing up the Machine-Level settings using LCSCmd.exe

Don’t forget to copy the XML files created by LCSCmd to another location, or else you still won’t have a backup if something terrible happens to your server.

Databases on Windows Server 2003

You may not already have backup software which allows you to backup databases using VSS (for example NTBackup can’t do this); this can be the case if you have decided to install OCS 2007 R2 on a Windows 2003 OS.  In that situation, you may decide to create a backup task using the SQL Server Management Studio (SSMS) Express Edition instead. The tool will not be installed by default, but you should  download and install it manually, since OCS 2007 R2 does use SQL Express. The tool is free and you can download it here.

Unfortunately, SQL Express doesn’t provide you with the option to create a scheduled backup, because it doesn’t have the SQL Server Agent, which is only available in a full SQL Server edition; as a result, we will need to create our own script. To create the script, open SSMS, right-click on one of your OCS databases and select the Tasks option, followed by Backup;  a new window will be opened:

1060-OCS3.JPG

Figure 3. Backing up a database with SSMS Express

In most cases, you could keep the defaults which are filled in when the Window opens. The only thing which you might want to change is whether you want to keep the previous backup (by appending data) or overwrite it. This can be controlled on the options tab:

1060-OCS4.JPG

Figure 4. Choosing to append new data to the previous backup in the ‘Options’ tab.

Once you are satisfied with the settings, click on the Script button and select the Script action to file option; this option will let you create a SQL script which creates the backup for you. Save the script file in a directory which is suitable for your organization; I created a backup folder, which in turn contains a script folder where all my backup scripts are located. You will have to create a script for each database that you want to backup, so in my case that would be:

  • RTCab
  • RTC

Now it’s time to create a batch file which will execute both SQL scripts using the SQLCMD SQL command line tool, which can be found in the Tools folder in the SQL Server installation folder. The batch file only contains two lines, one for each SQL script:

It doesn’t particularly matter where you save the batch file; it should be a logical place where you can find it again, such as the scripts directory which also contains the SQL scripts.

Before you can run the scripts, you might have to make some additional changes to the SQL Express configuration. The default SQL Express configuration won’t accept remote connections, including connections made with SQLCMD. Thankfully it’s not hard to configure SQL Express to accept remote connections, and you can find all the information you need to do that on the SQL Express blog. If you don’t follow the instructions there, then this the script won’t work.

Now we have all the files we need, we can create a scheduled task to run the script once a day. Let’s say we run the script each day at 23:00; this can be done by selecting the Create a basic task option, which will guide you through the whole process. Simply provide all the requested data, and when you click finish you will have created your own SQL backup schedule.

1060-OCS5.JPG

Figure 5. Scheduling your batch file to run daily at 23:00

You’ve probably noticed that I’ve not covered a method for backing up your fileshares using Windows Server 2003. This is because, unlike Windows Server 2008, the older version of the OS doesn’t have a native method for handling this. To backup your fileshares on Windows Server 2003, I’m afraid you either need to perform the backups manually, or invest in a 3rd party tool.

Databases (and File Shares) on Windows Server 2008

Of course, there are better ways to create a backup of the SQL databases, and one of them is using Windows Server Backup, which is a standard program of Windows 2008. Using Windows Server Backup, you can create backups of the SQL databases using the Volume Shadow Copy Service, and you won’t have to use the workaround of scripts and schedules which we’ve just created.

Because Windows Server Backup isn’t installed by default either, you’ll have to install it via one of the following two methods:

  • via “Add Features” in the Server Manager
  • using the servermanagercmd -install backup command

Once Windows Server Backup in installed, start the program and choose the option to Create a backup schedule. This will launch a wizard which will guide you through all the steps needed to create a successful backup.

1060-OCS6.JPG

Figure 6. The Windows Server Backup wizard

Another advantage of using Windows Server Backup is that you also have the option to create a backup of the complete server, containing all the data needed to recover the OCS server, except for the Active Directory aspects. When using the scripts we created earlier, you will have to backup everything (in particular, the file shares) using another backup program in order to safeguard your files. This is because all the files which the script creates will still be located on the local server, which is almost no back up at all, and also because those scripts will only backup the databases.

So, now we have created a backup of our databases, it’s time to backup our file shares. This can be either done by using your current 3rd party backup solution or NTBackup/Windows Server Backup if you don’t have another backup solution already in place.

As we have already installed Windows Server Backup on our OCS 2007 R2 server, we will continue to use this as our backup solution. There are two options at this stage:

  • Play it safe and create a full server backup of your OCS server
  • Just select the files which are needed for recovering your OCS server

The first option doesn’t require further explanation because you can’t really do much wrong there. The second option is a little bit trickier because you need to manually select the files which you want to backup.

So let’s start Windows Server Backup again and choose the Create a backup schedule option; as before, this will start a wizard which will help you setup the backup. Once you’ve selected the custom option instead of full server (which will be fairly early on in the process) you will have to specify which files you want to backup. In my environment, this will be the following folders:

  • C:\Program Files\Microsoft Office Communications Server 2007\Web Components\Data MCU Web\Web
  • C:\Program Files\Microsoft Office Communications Server 2007\Web Components\Data MCU Web\Non-Web
  • C:\Program Files\Microsoft Office Communications Server 2007\Web Components\Address Book Files
  • C:\Program Files\Microsoft Office Communications Server 2007\Application Host\Application Data
  • C:\Program Files\Microsoft Office Communications Server 2007\Web Components\AutoUpdate
  • C:\Program Files\Microsoft Office Communications Server 2007\Web Components\DeviceUpdateFiles

For all the folders you have selected, you will need to make sure, that if other folders exist inside them, these sub-folders are also selected to be backed up.

Active Directory

The last item we need to backup is your Active Directory, and this can be done by creating a backup of one of your domain controllers. As before, this process can either be done via the built-in backup software or with a 3rd party tool. There’s a lot of detail I could go into regarding backing up your Active Directory, and unfortunately that detail is way outside the scope of these articles. So, for more detailed information on how to tackle this, I would like to refer you to this TechNet site. It covers some of the same material as this article (specifically when discussion Windows Server Backup), but it covers Active Directory Snapshots towards the end.

Summary

So, now you know where all the data and settings for your OCS environment are being stored, and you know what you need to backup (and what’s not so important), and how to go about it. In most cases you’ll already have all the tools you need (if you’re using Windows Server 2008), but you might have to be prepared to spend some money on backup software if you don’t have a tool already.

While I covered a lot of ground here, hopefully you’ll see that this process isn’t actually that complicated. In the next article in this series, I will explain several possible disasters which might happen to your OCS environment, and how you can recover them as easily as possible. This will be everything from a file share which becomes corrupted, all the way over to a complete OCS server recovery. Hopefully you won’t ever have to worry about any of these, but you can’t be too safe.

This article was commissioned by Red Gate Software, engineers of ingeniously simple tools for optimizing your Exchange email environment.
Learn more about Exchange Server Archiver and PST Importer.