Run Redgate Monitor on PostgreSQL with TimescaleDB
I’m delighted to share that you can now run Redgate Monitor’s data repository on PostgreSQL with TimescaleDB, achieving dramatically improved performance and scalability at reduced cost.
Why are we doing this?
Until today, Redgate Monitor has relied on a SQL Server data store. That’s often a great choice, and the ability to run on SQL Server certainly isn’t going away, but this new capability can provide significant benefits:
Direct cost savings
SQL Server licensing costs represent a significant expense as part of operating the monitoring solution. Around a third of Redgate Monitor installations choose to run on SQL Server Enterprise Edition, but even Standard Edition can easily stretch into many tens of thousands of dollars or more.
PostgreSQL and the TimescaleDB extension are both free of any license fees, eliminating that spend and greatly reducing Redgate Monitor’s Total Cost of Ownership.
Built to scale
Monitoring tools like Redgate Monitor rely on extremely large volumes of “time series” data. That type of data has unique characteristics, and while in principle it can be stored happily in a relational database like SQL Server, a time series database allows for significant use-case-specific optimizations, making that the more natural choice.
TimescaleDB, built on PostgreSQL, is a true time series database, purpose built for working efficiently with exactly this kind of data. Across key metrics (CPU, I/O, storage space) we’ve measured significant improvements vs running Redgate Monitor on SQL Server, translating into superior performance and reduced hardware requirements.
Natural for the PostgreSQL DBA
For some organizations, PostgreSQL is the primary or only database platform being managed by Redgate Monitor. Until today they’ve had to run a SQL Server database just to run the monitoring solution, and while using a managed database like Azure SQL Database or Amazon RDS reduces the burden of that, it still feels unnatural for some.
This new capability makes it easier to run Redgate Monitor in PostgreSQL-heavy companies, which have limited SQL Server expertise available.
About TimescaleDB
TimescaleDB is the leading time series database. It’s a simple, free, PostgreSQL extension which adds time series capabilities to the PostgreSQL engine, while retaining all the key manageability benefits of PostgreSQL. That’s because it is a regular PostgreSQL installation, just with an extension installed – your approach to things like backups, high availability, security, and maintenance are all unchanged.
TimescaleDB powers millions of databases around the world, including at some of the world’s largest organizations. TimescaleDB is also the database used to back our own hosted version of Redgate Monitor, currently in preview.
What’s next?
This new capability is part of our commitment to make Redgate Monitor easier to operate at scale. Redgate Monitor’s users include teams managing some of the world’s largest and most complex database environments, across thousands of extremely busy instances – the larger the environment being monitored, the more valuable the cost and scalability benefits become.
To that end we’ll be making further performance improvements over the coming months, as well as allowing you to run Redgate Monitor’s components on Linux. We’ll also be exploring data migration paths, allowing you to move existing SQL Server-based Redgate Monitor installations to run on TimescaleDB instead.
For now, I’m really excited about the improvements offered by running Redgate Monitor in this way. You can read more about why we picked to run Redgate Monitor on TimescaleDB in an upcoming post, or see the documentation for details on how to get started running Redgate Monitor on TimescaleDB.
FAQ
Is the option to use SQL Server going away?
Absolutely not. We appreciate that this ability to run on PostgreSQL with TimescaleDB won’t be for everybody. If you’re happily running Monitor, we’re committed to retaining the ability to use SQL Server as the data store – this is a new option, not something you have to use.
Why not just run on vanilla PostgreSQL?
Redgate Monitor (like all monitoring solutions) is heavily dependent on storing time series data – that is, a sequence of measurements collected over time. TimescaleDB transforms PostgreSQL into a database purpose-built for that kind of time series data, leading to dramatically improved scalability vs native PostgreSQL.
We recognize that requiring TimescaleDB does impose an additional consideration, but believe in this case that the scalability benefits are worth that. You can read more about specifically why we chose TimescaleDB in an upcoming post.
Can I monitor SQL Server instances with a PostgreSQL TimescaleDB repository?
Yes, the choice of database to use as Redgate Monitor’s repository is entirely separate from what you monitor. So you can monitor SQL Server, PostgreSQL, Oracle, MySQL or MongoDB databases, regardless of whether the repository is hosted on SQL Server or TimescaleDB.
Can I switch an existing SQL Server-based installation to TimescaleDB?
It’s not possible to do this today. We do plan to offer a migration path later this year, allowing existing configuration and data history to be transferred to a new installation.
It’s worth knowing in the meantime that today, a Web Server can connect to any number of Base Monitor services, regardless of whether those services are backed by a SQL Server repository or by TimescaleDB (the database used behind the scenes is invisible to the Web Server). So you can create a new Base Monitor service backed by TimescaleDB and use that to monitor new servers, while keeping your previous SQL Server backed Base Monitor service for previously added instances.
How do I run PostgreSQL with TimescaleDB?
You can either operate it yourself by installing the extension on a Linux-based PostgreSQL instance, or can run it on Timescale Cloud.
How great are the performance and scalability benefits?
We’ll be publishing some benchmarking in due course, but we’ve seen multiple fold improvements across CPU, IOPS, storage, and page load times.
Which editions of Redgate Monitor support this?
Any edition of Redgate Monitor can use TimescaleDB as the repository.
You may also like
Blog