Migrating Between Monitoring Systems
This question comes up all the time: How do you migrate between monitoring systems?
The answer is both simple and complicated. In order to understand this better, rather than just rely on my own knowledge, I reached out to a number of people to see how they accomplished this. I’m going to summarize their process for you here in order to help others who may find themselves in need of this information.
Don’t Migrate The Data
One of the first things I would have thought about when someone says the word “migrate” would be the need to move the data between two monitoring systems. In case you’re not aware, every monitoring system has it’s own database design. So moving data between two, very, disparate systems is going to be a challenge. How did all the people who talked to me about migrating monitoring systems handle this?
They didn’t.
No one talked about even attempting to move the data between systems. Everyone talked about using your existing monitoring as a way to document the servers you need to monitor. It just makes sense that the servers you’re interested in with the original monitoring system will be the ones you need in the new. However, data movement just wasn’t something that anyone considered a necessary part of a monitoring system migration.
Side-By-Side Comparisons
A good number of people would put the two monitoring systems into a head-to-head comparison. TJay Belt said “Leave both running and try to find the same answer with the new one and compare it to the old “comfortable” one.” Greg Moore agreed, “I’d probably run them in parallel at first.”
The assumption everyone makes is that you’re moving between monitoring systems because either your old system was poor and you’ve identified the need for an upgrade, or, you have new requirements that the old system just can’t support. Either way, keeping both systems online for a period of time allowed time for training people in how the new system works. You can look at the old system that you’re familiar with and then compare metrics to the new system, just to be sure you can find what you need.
Tune The Alerts
Out of the box, any monitoring system probably doesn’t have the alerts set the way you need them. They may have things enabled that you don’t need. There could be alerts that are disabled that you’re pretty desperate to turn on. While a monitoring tool like Redgate SQL Monitor has a pretty good set of thresholds for the alerts, they’re not going to necessarily be perfect for what you need. You may also find that some metrics and alerts in the old system were customized and you’ll need to do the same in the new system.
John Sterrett said, “I would spin up the new monitoring system and have it running side by side to make sure it caught the alerts I expected.” He goes further, “I would most likely have the new system do alerting internally until it’s validated.” This is another reason to run things side by side.
Nigel Foulkes-Nock followed a similar process, focusing on ensuring that the alerts were configured the same way. He went into a little detail on the process:
“Review existing Alerts to see if any are useful / recently fired (based on my own knowledge and records within our call logging system that had been raised as a result of the monitoring tool. Document the alerts that are required and set them up on the replacement monitoring tool.”
Garry Bargsley is running the process right now and said, “My plan is to bring the new one online and see how the new monitor catches alerts. Then mirror alerts as pertinent.”
Everyone agreed, some time spent tuning the alerts as a part of the process of migrating between the systems is necessary.
Just Cut Over
More than a few people just implemented the new monitoring system and turned off the old one on the spot. Johan Bratt
Since no one was interested in migrating the data, it makes sense that one mechanism for migrating between two monitoring systems is to simply switch. As Greg and Josh put it, if it’s a tangled mess, why even use it as a reference.
Conclusion
Based on what everyone says, you would be best served by using the existing system, assuming it’s not a mess, to document how the new system should be configured. You can run them side-by-side for a time as a teaching tool. Also, having them both running lets you use alerting in one system to tune the alerts in the other. Identify any customization necessary by comparing the two, then, turn the old system off. Or, if the old system just isn’t cutting it, turn it off and turn on the new, better, monitoring tool. Now that you know how to make the move, please, take a moment to check out SQL Monitor.
Tools in this post
Redgate Monitor
Real-time SQL Server and PostgreSQL performance monitoring, with alerts and diagnostics