SQL Server R Services: Working with Data Frames

Although you can get started with R in SQL Server without understanding data frames, they are a key structure of the R language that are the equivalent of SQL Server table variables. They give you many ways of manipulating and analyzing data and passing it between R and SQL Server. For a database professional, they provide a clear and familiar concept when getting to grips with integrating R into the database.… Read more

Thoughts about submitting a new Power BI Quick Measure

‘Quick Measures’ is a feature included in Power BI Desktop April update. It’s still in preview, so you need to enable this feature in order to use it. Inside Power BI Desktop, click ‘File’->‘Options and Settings’->‘Settings’->‘Preview Features’ and check the box beside ‘Quick Measures’, as illustrated in the image below.   Microsoft created an online gallery … Read more

SQL Server R Services: Working with ggplot2 Statistical Graphics

It is when you use R in SQL Server with one of the huge range of packages that comes with it that you can begin to appreciate the power of the system. With a package such as ggplot there are many 'knobs one can twiddle' in order to get spectacular and informative visualisations. Rob Sheldon continues his beginners series for R in SQL Server by showing how to refine the output to get it as you need it.… Read more

Pseudonymization and the Inference Attack

It is surprising that so much can be identified by deduction from data. You may assume that you can safely distribute partially masked data for reporting, development or testing when the original data contains personal information. Without this sort of information, much medical or scientific research would be vastly more difficult. However, the more useful the data is, the easier it is to mount an inference attack on it to identify personal information. Phil Factor explains.… Read more

The Best of Both Worlds: Using Excel and Power BI Together

Excel and Power BI work well together. This allows you to use the two tools together to provide for many types of business workflow and BI practices. You can publish an Excel file to Power BI to share with others, analyse a Power BI dataset in Excel or import either an Excel workbook or Excel data to Power BI. You can gain the workgroup power and business-orientation of Power BI without losing the ease and versatility of Excel. Saurabh shows how.… Read more

SQL Server R Services: Digging into the R Language

It is not just the analytic power of R that you get from using SQL Server R Services, but also the great range of packages that can be run in R that provide a daunting range of graphing and plotting facilities. Robert Sheldon shows how you can take data held in SQL Server and, via SQL Server R Services, use an R package called ggPlot that offers a powerful graphics language for creating elegant and complex plots.… Read more

SQL Server R Services: The Basics

It is possible to do a great deal with R within SQL Server, but it is best to start by doing analysis in R on numeric data from SQL Server and returning the results to SQL Server. There is great value to be gained even with this basic foundation. Robert Sheldon is on hand to give you a kick start with the first in his series on beginning with R in SQL Server.… Read more

Filtering Errors in PowerBI

I just faced this problem when trying to analyze some information from google analytics in PowerBI. In my specific situation, I was trying to extract a number from some URL querystrings and filter the rows without the number. When I set the data type of the column to decimal, the rows without a number receive … Read more

Deploying, Securing, and Updating Power BI Reports

Before Excel and other data-linked spreadsheets, business reports needed to be hand-crafted by IT experts, and changes were arbitrated slowly via change control. Power BI was designed to take the liberation that Excel pioneered to the max, by allowing 'power' users to not only to create Power BI desktop reports, graphics and visualisations in Power BI App, but then link and publish these to the Power BI app. These in turn can be given row level security and have their underlying data kept up-to-date so that users can log in and see the relevant and appropriate data in a browser. Saurabh Desai shows how.… Read more

Statistics in SQL: Kendall’s Tau Rank Correlation

Statistical calculations in SQL are often perfectly easy to do. SQL was designed to be a natural fit for calculating correlation, regression and variance on large quantities of data. It just isn't always immediately obvious how. In the second of a series of articles, Phil factor shows how calculating a non-parametric correlation via Kendall's Tau or Spearman's Rho can be stress-free.… Read more

Scala and Apache Spark in Tandem as a Next-Generation ETL Framework

Scala and Apache Spark might seem an unlikely medium for implementing an ETL process, but there are reasons for considering it as an alternative. After all, many Big Data solutions are ideally suited to the preparation of data for input into a relational database, and Scala is a well thought-out and expressive language. Krzysztof Stanaszek describes some of the advantages and disadvantages of a scala-based approach to implementing and testing an ETL solution. … Read more

Comparing SSIS Catalog Contents Using DBFit Framework

When you are doing the rapid deployment of an updated SSIS project, there are a number of things you have to check to make sure that the deployment will be successful. These will include such settings as the values in environment variables, Package parameters and project parameters. The DbFit test framework turns out to be ideal for the purpose of doing final checks as part of a deployment process, as Nat Sundar demonstrates.… Read more

Generating Plots Automatically From PowerShell and SQL Server Using Gnuplot

When you are automating a number of tasks, or performing a batch of tests, you want a way of automating the production of your plots and graphs. Nothing beats a good graphical plot for giving the indications of how the process went. If you are using PowerShell and maybe also SQL Server, it pays to use a command-line plotting tool such as Gnuplot to do all the hard work. It turns out to be handy for a range of data jobs, turning PowerShell into a handy data science tool.… Read more

Using an R Package within SQL Server with Real time analysis in Power BI

We all know how easy it is to run R, together with dependent packages, from SQL Server on order to do statistical analysis, and then turn the result into a real-time graph using Power BI; but can you actually do it? Saurabh Desai gives a step-by-step hand-holding guide to turn even the most timid into a BI guru. There is nothing difficult in using R in SQL Server 2016 once you know the steps.… Read more

Using SQL SERVER and Cloud Hosted Databases with OBIEE 11g

If your organisation is committed to using Oracle Business Intelligence Enterprise Edition (OBIEE) 10g/11g /12c as their BI solution, you aren't thereby committed to using Oracle throughout your organisation. You can use a range of data sources including SQL Server, and save a great deal of money by doing so. Sadly, Oracle will only support the use of the venerable SQL Server 2008 R2. Zafar Ali demonstrates how to connect OBIEE to the world beyond Orac… Read more

Reconciling Data Across Systems Using a Reconciliation Hub

In many enterprises, where there are a number of separate systems engaged in processing data, there arises the daunting task of checking and reconciling data as it flows between systems. Discrepancies in data must be detected, tracked and corrected as quickly as possible: there is no room for error in doing this. A Reconciliation Hub can provide the answer, as Rahul Gupta explains.… Read more

Using Power BI Desktop to Visualize SQL Server Metadata

You can easily use PowerBI Desktop to show graphically how your database is growing, which tables are taking the most space, how various parts of SQL Server is consuming memory, its use of indexes and so on. Sure, you can create graphs in SSMS, but with PowerBI, you can create reports that you can then publish to others, and which allow drill-down. It is a great way to get familiar with PowerBI Desktop as well. Rob Sheldon shows how simple it is to do.… Read more