09 April 2019
09 April 2019

Processing Data Using Azure Data Lake, Azure Data Analytics, and Microsoft’s Big Data Language U-SQL

Data analytics has become one of the powerful domains in the world of data science. An enormous amount of data is being generated by each organization in every sector. Computer science has found solutions to store and process this data in a smart way through a distributed file system. One such example is Azure Data Lake. It uses the Hadoop Distributed File System, and to perform analytics on this data, Azure Data Lake storage is integrated with Azure Data Analytics Service and HDInsight. In this article, Suhas Pande will explain how to store data using Azure Data Lake and how to perform data analysis on it using U-SQL, a big data SQL and C# language. … Read more
03 April 2019
03 April 2019

SQL For Cosmos DB – Tips and Tricks

Building on the introduction to the SQL used by Cosmos DB in the first article in the series, here you will learn how to handle some of the more classic challenges that you may face when querying JSON documents. Adam Aspin shows you some of the workarounds that are useful in practice when finding and shaping output data ready for further analysis.… Read more
11 July 2018
11 July 2018

Introduction to Azure Cosmos DB Emulator for Creating Applications

Azure Cosmos DB is Microsoft’s NoSQL database platform running in the cloud. In this article, Suhas Pande explains many of the core concepts in Cosmos DB. Additionally, he goes over how to set up a local Cosmos DB emulator to create collections and documents. Using a local emulator is free and allows development with Cosmos DB without being connected to Azure. … Read more
18 October 2017
18 October 2017

Questions About Amazon Data Migration Service (AWS DMS) That You Were Too Shy to Ask

Can you imagine it? You are in a group of smart database people, and they are debating the finer points about AWS DMS, and you don't even know what the letters stand for. You just feel too shy to ask those basic questions that seem ridiculous once you're up to speed. Laerte Junior answers all the questions you need answers for when facing the prospect of getting familiar with Amazon's useful Database Migration Servic… Read more
14 September 2017
14 September 2017

The Quick and the Dead Slow: Importing CSV Files into Azure Data Warehouse

Although Azure Data Warehouse is part of the bright new jewellery of the Microsoft Data Platform, the old Data Warehouse rules still apply where data imports are concerned. When it comes to data import, it pays to choose the fastest import method first and prepare your data first to ensure that it is compatible with your choice. The subtlety is in the details, as Feodor explains.… Read more
13 September 2017
13 September 2017

Questions About RDS SQL Server That You Were Too Shy to Ask

There are a number of different ways that you can host SQL Server. RDS SQL Server, for example, uses SQL Server within AWS as a simple database service, much like a more versatile alternative to MySQL. Obviously, it is a compromise, in that you lose many of the extras beyond the database. Laerte Junior answers those questions about RDS that people seem to assume you know the answers to, but which you may be too shy to ask.… Read more
18 August 2017
18 August 2017

Using the Copy Wizard for the Azure Data Factory

Creating a feed for a data warehouse used to be a considerable task. Now, it just takes a few minutes to work through a series of screens that, in this example, create a pipeline that brings data from a remote FTP server, decompresses the data and imports the data in a structured format, ready for data analysis. The Copy Wizard for the Azure Data Factory is a great time-saver, as Feodor Georgiev explains.… Read more
01 August 2017
01 August 2017

Pseudonymization and the Inference Attack

It is surprising that so much can be identified by deduction from data. You may assume that you can safely distribute partially masked data for reporting, development or testing when the original data contains personal information. Without this sort of information, much medical or scientific research would be vastly more difficult. However, the more useful the data is, the easier it is to mount an inference attack on it to identify personal information. Phil Factor explains.… Read more
28 July 2017
28 July 2017

Data in Motion and Data at Rest

Microsoft (StreamInsight), and Azure Stream Analytics represent a very different model for processing data. They are concerned with processing complex event streams of data (CEPs) from such things as sensors to deduce significant patterns and apply filters. Joe Celko discusses the background to an intriguing technology of complex event processing to establish the difference between data at rest, and data on the move.… Read more
27 June 2017
27 June 2017

Automating the Synchronization of RDS SQL Server Agent Jobs in a Multi-AZ Environment

Although Azure is the obvious Cloud service to host SQL Server, Amazon Relational Database Service (RDS) for SQL Server is a good choice when your organisation uses AWS. RDS deals with maintenance and monitoring, and supports the use of PowerShell to automate routine tasks. What if a script needs to be triggered by an unscheduled event? Even in this case, RDS can be configured to run scripts to react when something like a failover happens. Laerte Junior shows how easy it is to set up Lambda functions and some PowerShell scripts to automatically synchronise agent jobs after a failover.… Read more
22 May 2017
22 May 2017

Getting What You Need From Azure Storage Disks

If you need persistent data disks for Azure IaaS VMs that are supported on both Windows and Linux then you will be interested in Azure Storage Disks. These can increase the storage capacity of your VMs by up to a terabyte per disk, and they not only allow several availability options, but also offer a range of performance in terms of I/O throughput and latency. With right configuration, you can create as much of the right sort of storage as you need.… Read more
10 May 2017
10 May 2017

Personal Data, Privacy, and the GDPR

Now that there have been well-publicised examples of the awful consequences of data breaches and data misuse, there is increasing public pressure for legislation on privacy and personal data that has enough clout to prosecute serious offenders. In the vanguard has been the EU data protection regulation, soon to be succeeded by the GDPR. It defines IT practices for data that are likely to extend worldwide. William Brewer gives a rundown of what he sees as the implication for IT practice.… Read more
25 April 2017
25 April 2017

Azure Load Balancers and SQL Server

Load balancing in Azure has more importance for the DBA, because it is essential for Windows Server Failover Clustering in Azure, whether it is for AlwaysOn Availaiblity Groups, Failover Clustered Instances, or any other highly-available solution. Azure load balancing works out the location of the availability group, and routes traffic there. The load balancer detects a failure, and routes traffic to the new primary replica. Joshua Feierman gives an overview of what is required.… Read more
25 April 2017
25 April 2017

How to Secure Your Azure Storage Infrastructure

Azure storage is an essential foundation for the more sophisticated services that Microsoft Azure provides. It is therefore important to understand how to make access to your data in Azure storage secure, to control access appropriately, to log activity and to get metrics on usage. Security in Azure can be easily managed and controlled via policies. There are a variety of ways to achieve the types of control over access that your applications need, as Christos Matskas explains.… Read more