By need

By role

Phil Factor

Phil Factor

Phil Factor (real name withheld to protect the guilty), aka Database Mole, has 30 years of experience with database-intensive applications. Despite having once been shouted at by a furious Bill Gates at an exhibition in the early 1980s, he has remained resolutely anonymous throughout his career. See also :

Follow on
Phil Factor
24 August 2017
Phil Factor
24 August 2017

When is the Data Deleted?

3
6
Imagine that your business is providing a service to individuals, and you charge by the amount of usage. You are trading your service internationally and need to keep a record of who among your customers does what. You then produce invoices and keep accounts. Your customers pay via a third-party service. So far so good. … Read more
3
6
Phil Factor
03 August 2017
Phil Factor
03 August 2017

Statistics in SQL: Student’s t-test

0
6
Many undergraduates have misunderstood the name 'Students' in the t-test to imply that it was designed as a simple test suitable for students. In fact it was William Sealy Gosset, an Englishman publishing under the pseudonym Student, who developed the t-test and t distribution in 1908, as a way of making confident predictions from small sample sizes of normally-distributed variables. As Gosset's employer was Guinness, the brewer, Phil Factor takes a sober view of calculating it in SQL.… Read more
0
6
Phil Factor
01 August 2017
Phil Factor
01 August 2017

Pseudonymization and the Inference Attack

0
10
It is surprising that so much can be identified by deduction from data. You may assume that you can safely distribute partially masked data for reporting, development or testing when the original data contains personal information. Without this sort of information, much medical or scientific research would be vastly more difficult. However, the more useful the data is, the easier it is to mount an inference attack on it to identify personal information. Phil Factor explains.… Read more
0
10
Phil Factor
23 June 2017
Phil Factor
23 June 2017

Is It Time To Stop Using IsNumeric()?

5
20
The old system function IsNumeric() often causes exasperation to a developer who is unfamiliar with the quirks of Transact SQL. It seems to think a comma or a number with a 'D' in the midde of it is a number. Phil Factor explains that though IsNumeric has its bugs, it real vice is that it doesn't tell you which of the numeric datatypes the string parameter can be coerced into, and because it doesn't check for overflow. Phil comes to the rescue with a couple of useful alternatives, one of which works whatever version of SQL Server you have, and which tell you what datatype the string can be converted to.… Read more
5
20
Phil Factor
07 June 2017
Phil Factor
07 June 2017

How to Automatically Create and Refresh Development and Test Databases using SQL Clone and SQL Toolbelt

2
7
In order to be able to deliver database changes more quickly, there are several tasks that must be automated. It can be a daunting job to ensure that the whole team has the latest database build when there is a proliferation of copies, and the database is big. Phil illustrates a solution by taking a set of Redgate tools to show how they can be used together, via PowerShell, to build a database from object-level source, stock it with data, document it, and then provision any number of test and development servers with the database build, taking care to save any DDL changes to the existing copies of the database.… Read more
2
7
Phil Factor
01 June 2017
Phil Factor
01 June 2017

Statistics in SQL: The Kruskal–Wallis Test

0
4
Before you report your conclusions about your data, have you checked whether your 'actionable' figures occurred by chance? The Kruskal-Wallis test is a safe way of determining whether samples come from the same population, because it is simple and doesn't rely on a normal distribution in the population. This allows you a measure of confidence that your results are 'significant'. Phil Factor explains how to do it.… Read more
0
4
Phil Factor
23 May 2017
Phil Factor
23 May 2017

SQL Server User-Defined Functions

2
12
User-Defined Functions (UDFs) are an essential part of the database developers' armoury. They are extraordinarily versatile, but just because you can even use scalar UDFs in WHERE clauses, computed columns and check constraints doesn't mean that you should. Multi-statement UDFs come at a cost and it is good to understand all the restrictions and potential drawbacks. Phil Factor gives an overview of User-defined functions: their virtues, vices and their syntax.… Read more
2
12
Phil Factor
05 May 2017
Phil Factor
05 May 2017

Visual Checks on How Data is Distributed in SQL Server

1
6
There are many reasons for wanting to know how data is distributed. Sometimes you just want a rough idea of the way that data is distributed in a column. You may think, wouldn't it be nice to have a SQL function that just showed you roughly what the distribution was, graphically, in the results pane. Phil Factor thought that was well and turned the vague wish into reality.… Read more
1
6
Phil Factor
13 April 2017
Phil Factor
13 April 2017

Statistics in SQL: Simple Linear Regressions

3
13
Although linear regressions can get complicated, most jobs involving the plotting of a trendline are easy. Simple Linear Regression is handy for the SQL Programmer in making a prediction of a linear trend and giving a figure for the level probability for the prediction, and what is more, they are easy to do with the aggregation that is built into SQL.… Read more
3
13
Phil Factor
10 April 2017
Phil Factor
10 April 2017

Statistics in SQL: Kendall’s Tau Rank Correlation

2
7
Statistical calculations in SQL are often perfectly easy to do. SQL was designed to be a natural fit for calculating correlation, regression and variance on large quantities of data. It just isn't always immediately obvious how. In the second of a series of articles, Phil factor shows how calculating a non-parametric correlation via Kendall's Tau or Spearman's Rho can be stress-free.… Read more
2
7
Phil Factor
27 March 2017
Phil Factor
27 March 2017

Generating Plots Automatically From PowerShell and SQL Server Using Gnuplot

0
8
When you are automating a number of tasks, or performing a batch of tests, you want a way of automating the production of your plots and graphs. Nothing beats a good graphical plot for giving the indications of how the process went. If you are using PowerShell and maybe also SQL Server, it pays to use a command-line plotting tool such as Gnuplot to do all the hard work. It turns out to be handy for a range of data jobs, turning PowerShell into a handy data science tool.… Read more
0
8
Phil Factor
23 February 2017
Phil Factor
23 February 2017

Doing Fuzzy Searches in SQL Server

6
17
A series of arguments with developers who insist that fuzzy searches or spell-checking be done within the application rather then a relational database inspired Phil Factor to show how it is done. When the database must find relevant material from search terms entered by users, the database must learn to expect, and deal with, both expected and unexpected … Read more
6
17
Phil Factor
14 February 2017
Phil Factor
14 February 2017

String Comparisons in SQL: The Metaphone Algorithm

7
5
When exploring the use of the Metaphone algorithm for fuzzy search, Phil couldn't find a SQL version of the algorithm so he wrote one. The Metaphone algorithm is built in to PHP, and is widely used for string searches where you aren't always likely to get exact matches, such as ancestral research and historical documents. It is particularly useful when comparing strings word-by-word. With a SQL version, it is easy to experiment on large quantities of data!… Read more
7
5
Phil Factor
02 February 2017
Phil Factor
02 February 2017

Metaprogramming

25
5
Over the history of personal computing, it seems that the best software was written by a team of five or fewer programmers. CP/M, the first PC operating system was built by one guy, Gary Kildall, though he got parts from others such as Gordon Eubanks. MSDOS originated in QDOS, again written by one person, Tim … Read more
25
5
Phil Factor
01 December 2016
Phil Factor
01 December 2016

Lists With, or Without, Ranges in both T-SQL and PowerShell

5
6
Whether you are working in a procedural language like PowerShell or in T-SQL, there is something slightly bothersome about having to deal with parameters that are lists, or worse with ranges amongst the values. In fact, once you have a way of dealing with them, they can be convenient, especially when bridging the gulf between application and the database. Phil Factor shows how to deal with them.… Read more
5
6
Phil Factor
03 November 2016
Phil Factor
03 November 2016

Using SQLite with PowerShell and SQL Server

2
9
When you combine PowerShell and SQLite, you can perform powerful magic. Phil Factor is in awe of SQLite and gives a brief demonstration of how easy it is to use. Just to encourage anyone who is unfamiliar with the database, he includes a giant-sized SQLite version of the old PUBS database that the first generation of RDBMS developers cut their teeth on. … Read more
2
9