Managing an IT team is a tough job. While demands on your services are increasing, both internally and externally, the need to be aware of and in control of application and server performance is a constant pressure. In fact, there are four key elements that currently affect the working life of an IT leader:
1. Server estates are growing
The demand for data storage, analysis and utilization continues to grow, and to meet that demand more databases and servers are required. Redgate’s 2019 State of SQL Server Monitoring survey revealed that estates have grown by an average of 4% of the last 12 months, for example, with the largest estates growing the fastest.
As well as knowing how much space is required, IT leaders need to understand what is being stored and where, and have important decisions to make over what needs to be stored, for how long, and who has access to it.
To meet the demands of development teams, and follow good DevOps practices, you also need to be able to provide dev and test data environments with data that closely matches production without compromising security or confidentiality.
2. Server estates are becoming more diverse
While Redgate’s monitoring survey showed that SQL Server is still the most used database management system, the diversity of server estates is increasing. The survey revealed that Oracle usage has gone up by around 20% in the past year, and occasional usage of platforms like PostgreSQL and MongoDB has also risen.
One reason could be that many organizations are now using different platforms for different needs. For example, Facebook uses MySQL as its main data storage system to handle information such as user details and posts, but uses Haystack for photo storage due to the rapid expansion capabilities needed for the millions of new photos users upload each week. For its search functionality, it then turns to Apache Cassandra because of the high availability and superior performance required.
As performance optimization is pushed from an ever more demanding user base, the need to use the right platform for the right jobs is becoming key.
With the demand for more servers on more platforms increasing, the ability to spin these up (and down) in the cloud becomes an easier route than on-premises hosting. In fact, there has been a 15% increase in people using SQL Server in the cloud for at least part of their estate in the last year, with the use of hybrid estates becoming ever more common. For example, it may be that organizations chooses to retain sensitive data on on-premises servers, while data that is not classed as personally identifiable information (PII) is located in the cloud.
With your data spread across different server platforms in different locations in such a way, maintaining an overview of estate health becomes a lot more difficult.
As a result of all these changes, migrations are also becoming a headache. From ensuring servers are on the latest version, to transferring data to cloud platforms, respondents to the survey revealed that migrating servers will be their biggest challenge this year.
3. Speed of delivery is the cornerstone of operations
With a growing, successful business, software delivery is at the heart of your ability to respond to customer needs and move ahead of the competition. You can’t afford to let the database get in the way when research from companies like Salesforce shows that 63% of customers now expect companies to provide new products or services more frequently than ever before.
According to Redgate’s 2019 State of Database DevOps survey, however, a wide range of stakeholders are involved in implementing a DevOps initiative, from IT Directors or Managers to Developers and the C-level, all of whom need to understand the benefits to be gained. They also need to understand that, without the right processes or tooling in place, database change management slows things down, adding risk, uncertainty, and blocking development and operations from working together to deliver value.
But while speed is important, it doesn’t come without risks.
4. Remaining compliant keeps you up at night
‘Human error’ was reported as the number one cause of database issues in 2019, and this can bring about a myriad of problems that range from poor performance through to compliance breaches.
The risk of a compliance breach brings about the danger of reputational damage alongside any fines that may come from regulators, which can be hefty. And breaches don’t just happen on production environments. Your dev and test servers can be just as vulnerable, and can often be a secondary thought when thinking about security practices.
So joined up development and operations that keep servers secure, reduce the risk of data breaches but enable Agile development processes is key. In fact our State of Database DevOps survey found that 61% of organizations think Database DevOps has a positive impact on meeting regulatory and compliance requirements
Keeping on top of this can be hard
So you’ve got a growing, diverse server estate that needs to enable fast service delivery while remaining compliant.
You need to know where your customer data lives and who has access to it for compliance reasons. At the same time, you need to speed up the development process with quick access to production-like data sets that don’t expose customer data.
It’s important your team know about, and can resolve, performance issues before they impact on your customer’s experience. You also need to ensure continued availability of services and access to data to meet compliance and regulatory needs.
The traditional approach is to use scripts to manually monitor your server status. Sure, you could hire an army of DBAs to do this, if only you could spin up database experts as quickly as you can spin up cloud servers.
The reality is that manual monitoring of your estate leaves you blind, vulnerable and slow. Your team aren’t aware of potential issues, and instead have to react when they occur. In doing this, they then spend a lot of time going through logs and reports to identify where and how the problem occurred, before spending even more time working out how to fix it.
Meanwhile, a vulnerable server could be leaking PII data without you knowing it, so that the first indication you get that something is wrong is the regulator knocking on your door with a hefty fine.
With monitoring, you’re in control
When you proactively monitor your estate, you can be alerted to and resolve performance issues quicker, before they impact your business. You also increase the visibility of the impact of deployments on server performance and availability and ensure an effective feedback loop between teams is maintained.
At the highest level, a single pane of glass view gives you the immediate status of all of your servers, regardless of where they are and how they are hosted, and your team has at their fingertips the tools to drill down, find and fix issues. And that’s not all. Powerful features like capacity planning, backup logs and security breach alerting mean you can always be on top of your server estate.
When you combine this with the provisioning of masked data that allows teams to work on realistic but secure copies of your production database, and automated deployments, you create an ecosystem that enables you to confidently manage your entire estate while delivering a speedy yet secure experience for your customers.
Also in Database development
What are the biggest challenges in monitoring SQL Server, and what should you expect from a third-party tool?
Redgate has just published the 2019 State of SQL server Monitoring Report, so I thought I’d read it to find out if the results it reveals chime with my own experience talking to large, distributed e...
Also in Blog
There's been a lot of excitement about the book Accelerate, which summarizes research from the past several years of the State of DevOps Report from DORA (which Redgate sponsors).
Perhaps the most po...
Also in DevOps
Ahead of our attendance at IDC’s DevOps Conference in September 2019, we approached Jen Thomson, IDC’s Senior Research Director & Lead for Accelerated App Delivery Practice, to help us articul...
Also about SQL Monitor
In 2018 we launched the industry’s first ever report into the state of SQL Server monitoring. The responses from over 600 SQL Server professionals gave us a unique insight into how they monitor thei...
Also about The State of SQL Server Monitoring
In 2018 we ran the industry’s first ever state of SQL Server monitoring survey. We received an excellent response from over 600 people in a range of sectors, in organizations of every size across th...
Also about monitoring database growth
If your SQL Server runs out of disk space, and it is running a database for an enterprise's trading application, then the company can't take money until the DBA fixes the problem. Even the worry of th...
Also about scaling monitoring
Back in 2012, I was transitioning into the role of 'Director of Data Services', at the Fortune 500 insurance organization where I worked. It wasn't that easy. Previously, as a DBA, I'd spent time de...