You can produce HTML from SQL because SQL Server has built-in support for outputting XML, and HTML is best understood as a slightly odd dialect of XML that imparts meaning to predefined tags. There are plenty of edge cases where an HTML structure is the most obvious way of communicating tables, lists and directories. Where data is hierarchical, it can make even more sense. William Brewer gives a simple introduction to a few HTML-output techniques.… Read more
Although linear regressions can get complicated, most jobs involving the plotting of a trendline are easy. Simple Linear Regression is handy for the SQL Programmer in making a prediction of a linear trend and giving a figure for the level probability for the prediction, and what is more, they are easy to do with the aggregation that is built into SQL.… Read more
Statistical calculations in SQL are often perfectly easy to do. SQL was designed to be a natural fit for calculating correlation, regression and variance on large quantities of data. It just isn't always immediately obvious how. In the second of a series of articles, Phil factor shows how calculating a non-parametric correlation via Kendall's Tau or Spearman's Rho can be stress-free.… Read more
Of course we all like our colleagues to think that we know everything there is to know about SQL Server Collations. However, the truth is that it is a rather complicated topic to fully understand and the cost of getting collation wrong can be great. If only one could ask certain questions on forums or at conferences without blushing. Help is at hand, because Robert Sheldon once again makes the complicated seem simple by answering those questions that you were too shy to ask… Read more
Scala and Apache Spark might seem an unlikely medium for implementing an ETL process, but there are reasons for considering it as an alternative. After all, many Big Data solutions are ideally suited to the preparation of data for input into a relational database, and Scala is a well thought-out and expressive language. Krzysztof Stanaszek describes some of the advantages and disadvantages of a scala-based approach to implementing and testing an ETL solution. … Read more
Some people will assure you that you can't do any serious statistical calculations in SQL. In the first of a series of articles, Phil factor aims to prove them wrong by explaining how easy it is to calculate Pearson's Product Moment Correlation.… Read more
When you are doing the rapid deployment of an updated SSIS project, there are a number of things you have to check to make sure that the deployment will be successful. These will include such settings as the values in environment variables, Package parameters and project parameters. The DbFit test framework turns out to be ideal for the purpose of doing final checks as part of a deployment process, as Nat Sundar demonstrates.… Read more
Entity Framework (EF) is Microsoft’s Object/Relational (ORM) database access library, with a new generation, EF Core, released in 2016. In this article Jon P Smith looks at six different software principles and patterns that help to keep the EF code nicely separated from the rest of the application. The six approaches make the EF database access code is easier to write, test, refactor and, most importantly, performance-tune.… Read more
When you are automating a number of tasks, or performing a batch of tests, you want a way of automating the production of your plots and graphs. Nothing beats a good graphical plot for giving the indications of how the process went. If you are using PowerShell and maybe also SQL Server, it pays to use a command-line plotting tool such as Gnuplot to do all the hard work. It turns out to be handy for a range of data jobs, turning PowerShell into a handy data science tool.… Read more
The network is important to any DBA because so much performance is dependent on I/O, because of the importance of security, and ensuring that everyone get the right access. DBAs generally need not become experts in Azure networks, but it helps to understand the concepts and language. If you are running a SQL Server Virtual Machine in Azure, then VNets, Subnets, Network Security Groups, VNet peering and VPN gateways are all worth knowing about in order to to keep SQL Servers running smoothly.… Read more
It is ironic that the users of database application need to rely on the very technologists that created the system to then devise and run their acceptance tests. Surely someone has devised a test system for databases that is simple enough for ordinary tech-savvy people to use and for them to create the tests? Yes they have. Fitnesse DbFit is a mature product that can, and does, test SQL Server databases, and Nat Sundar explains how to set it up and do it.… Read more
Dynamic Data Masking is a good way of rendering data unreadable for such purposes as user-acceptance testing, or demonstrating an application. It doesn't encrypt the data, and a knowledgeable SQL user can defeat it. However it provides a simple way to administer from the database what data the various users of a database application can and can not see, making it a useful tool for the developer.… Read more
I accidentally locked myself out of my house this evening, and so I had to climb up the wall like a burglar and clamber in through a bathroom window, while the neighbourhood kids all watched and giggled, probably taking secret photos to share on Snapchat or something. Embarrassing, yes, but what’s it got to do with Oracle? Well, the point… Read more
We all know how easy it is to run R, together with dependent packages, from SQL Server on order to do statistical analysis, and then turn the result into a real-time graph using Power BI; but can you actually do it? Saurabh Desai gives a step-by-step hand-holding guide to turn even the most timid into a BI guru. There is nothing difficult in using R in SQL Server 2016 once you know the steps.… Read more
When there are several SSIS projects with packages in a SQL Server Database or Data Warehouse development, automated deployments as part of Continuous Integration can get tricky. Nat Sundar describes how he created a Deployment script that is intended to provision a Data Warehouse for System Integrated testing (SIT).… Read more
Managed Disks have simplified way that Azure storage interacts with the users' virtual machines, thanks to the way that it eliminates the need to deal with the Storage Account. It is now easier to add new disks to a virtual machine, either in PowerShell or via the portal. The Storage Spaces feature in Windows Server can be used to aggregate disks together and obtain higher levels of performance. Joshua Feierman explains how to do it all.… Read more
A series of arguments with developers who insist that fuzzy searches or spell-checking be done within the application rather then a relational database inspired Phil Factor to show how it is done. When the database must find relevant material from search terms entered by users, the database must learn to expect, and deal with, both expected and unexpected … Read more
The ability to scale out SSRS, so that multiple report server instances can access a single report server database, is an Enterprise Edition-only feature. So when Rodney Landrum was tasked with implementing a scale-out solution, with failover, for SSRS on SQL Server Standard Edition, it required some creative thinking. This article describes his solution, its merits and drawbacks.… Read more
When exploring the use of the Metaphone algorithm for fuzzy search, Phil couldn't find a SQL version of the algorithm so he wrote one. The Metaphone algorithm is built in to PHP, and is widely used for string searches where you aren't always likely to get exact matches, such as ancestral research and historical documents. It is particularly useful when comparing strings word-by-word. With a SQL version, it is easy to experiment on large quantities of data!… Read more
Tables that return the value of the data in the table at a particular point of time have been with us since the first relational database, but have always required special queries and constraints, and can be tricky to get right. System-versioned Temporal Tables, new in SQL Server 2016, make such tables behave like any other. How do you create one, or modify an existing table? How can you get an In-Memory Optimized OLTP table to be Temporal? Alex Grinberg shows how. … Read more
