Managing an IT team is a tough job. While demands on your services are increasing, both internally and externally, the need to be aware of and in control of application and server performance is a constant pressure. In fact, there are four key elements that currently affect the working life of an IT leader:
1. Server estates are growing
The demand for data storage, analysis and utilization continues to grow, and to meet that demand more databases and servers are required. Redgate’s 2020 State of Database Monitoring survey revealed that estates are continuing to grow, with larger estates seeing an increase of 9% year on year.
As well as knowing how much space is required, IT leaders need to understand what is being stored and where, and have important decisions to make over what needs to be stored, for how long, and who has access to it.
To meet the demands of development teams, and follow good DevOps practices, you also need to be able to provide dev and test data environments with data that closely matches production without compromising security or confidentiality.
2. Server estates are becoming more diverse
While Redgate’s monitoring survey showed that SQL Server is still the most used database management system, the diversity of server estates is increasing. The survey revealed that MySQL and Oracle usage is increasing, and PostgreSQL is expecting growth in the next 12 months.
One reason could be that many organizations are now using different platforms for different needs. For example, Facebook uses MySQL as its main data storage system to handle information such as user details and posts, but uses Haystack for photo storage due to the rapid expansion capabilities needed for the millions of new photos users upload each week. For its search functionality, it then turns to Apache Cassandra because of the high availability and superior performance required.
As performance optimization is pushed from an ever more demanding user base, the need to use the right platform for the right jobs is becoming key.
With the demand for more servers on more platforms increasing, the ability to spin these up (and down) in the cloud becomes an easier route than on-premises hosting. In fact, there has been a 15% increase in people using SQL Server in the cloud for at least part of their estate in the last year, with the use of hybrid estates becoming ever more common. For example, it may be that organizations chooses to retain sensitive data on on-premises servers, while data that is not classed as personally identifiable information (PII) is located in the cloud.
With your data spread across different server platforms in different locations in such a way, maintaining an overview of estate health becomes a lot more difficult.
As a result of all these changes, migrations are also becoming a headache. From ensuring servers are on the latest version, to transferring data to cloud platforms, respondents to the survey revealed that migrating servers will be one of their biggest challenges this year.
3. Speed of delivery is the cornerstone of operations
With a growing, successful business, software delivery is at the heart of your ability to respond to customer needs and move ahead of the competition. You can’t afford to let the database get in the way when research from companies like Salesforce shows that 63% of customers now expect companies to provide new products or services more frequently than ever before.
According to Redgate’s 2020 State of Database DevOps report, however, a wide range of stakeholders are involved in implementing a DevOps initiative, from IT Directors or Managers to Developers and the C-level, all of whom need to understand the benefits to be gained. They also need to understand that, without the right processes or tooling in place, database change management slows things down, adding risk, uncertainty, and blocking development and operations from working together to deliver value.
But while speed is important, it doesn’t come without risks.
4. Remaining compliant keeps you up at night
‘Human error’ was reported as the number one cause of database issues in 2020, and this can bring about a myriad of problems that range from poor performance through to compliance breaches.
The risk of a compliance breach brings about the danger of reputational damage alongside any fines that may come from regulators, which can be hefty. And breaches don’t just happen on production environments. Your dev and test servers can be just as vulnerable, and can often be a secondary thought when thinking about security practices.
So joined up development and operations that keep servers secure, reduce the risk of data breaches but enable Agile development processes is key. In fact, our State of Database DevOps survey found that 67% of organizations think Database DevOps has a positive impact on meeting regulatory and compliance requirements
Keeping on top of this can be hard
So you’ve got a growing, diverse server estate that needs to enable fast service delivery while remaining compliant.
You need to know where your customer data lives and who has access to it for compliance reasons. At the same time, you need to speed up the development process with quick access to production-like data sets that don’t expose customer data.
It’s important your team know about, and can resolve, performance issues before they impact on your customer’s experience. You also need to ensure continued availability of services and access to data to meet compliance and regulatory needs.
The traditional approach is to use scripts to manually monitor your server status. Sure, you could hire an army of DBAs to do this, if only you could spin up database experts as quickly as you can spin up cloud servers.
The reality is that manual monitoring of your estate leaves you blind, vulnerable and slow. Your team aren’t aware of potential issues, and instead have to react when they occur. In doing this, they then spend a lot of time going through logs and reports to identify where and how the problem occurred, before spending even more time working out how to fix it.
Meanwhile, a vulnerable server could be leaking PII data without you knowing it, so that the first indication you get that something is wrong is the regulator knocking on your door with a hefty fine.
With monitoring, you’re in control
When you proactively monitor your estate, you can be alerted to and resolve performance issues quicker, before they impact your business. You also increase the visibility of the impact of deployments on server performance and availability and ensure an effective feedback loop between teams is maintained.
At the highest level, a single pane of glass view gives you the immediate status of all of your servers, regardless of where they are and how they are hosted, and your team has at their fingertips the tools to drill down, find and fix issues. And that’s not all. Powerful features like capacity planning, backup logs and security breach alerting mean you can always be on top of your server estate.
When you combine this with the provisioning of masked data that allows teams to work on realistic but secure copies of your production database, and automated deployments, you create an ecosystem that enables you to confidently manage your entire estate while delivering a speedy yet secure experience for your customers.
Also in Blog
The terms Continuous Integration and Continuous Deployment tend to be combined into the acronym CI/CD, often without any distinction between the two. CI and CD are distinct processes, even if ...
Also in DevOps
Since the release of the State of DevOps report by Puppet Labs ten years ago, various organizations have looked at how we could build better software. The generally agreed finding is the need to deliv...
Also in Database development
In 2018 and 2019 we ran the industry’s only surveys into the state of SQL Server monitoring, both of which provided unique insights into how companies and organizations monitor their estates, the te...
Also about SQL Monitor
You may think of monitoring as helping you keep your environments safe in times of technical crisis. But did you know, monitoring helps your teams work together in times of human-based crisis as well....
Also about The State of SQL Server Monitoring
In 2018 we launched the industry’s first ever report into the state of SQL Server monitoring. The responses from over 600 SQL Server professionals gave us a unique insight into how they monitor thei...
Also about monitoring database growth
If your SQL Server runs out of disk space, and it is running a database for an enterprise's trading application, then the company can't take money until the DBA fixes the problem. Even the worry of th...
Also about scaling monitoring
Back in 2012, I was transitioning into the role of 'Director of Data Services', at the Fortune 500 insurance organization where I worked. It wasn't that easy. Previously, as a DBA, I'd spent time de...