04 October 2016
04 October 2016

Why Would I Ever Need to Partition My Big ‘Raw’ Data?

0
20
Whether you are running an RDBMS, or a Big Data system, it is important to consider your data-partitioning strategy. As the volume of data grows, so it becomes increasingly important to match the way you partition your data to the way it is queried, to allow 'pruning' optimisation. When you have huge imports of data to consider, it can get complicated. Bartosz explains how to get things right; not perfect but wisely.… Read more
16 September 2016
16 September 2016

How to Start Big Data with Apache Spark

0
14
It is worth getting familiar with Apache Spark because it a fast and general engine for large-scale data processing and you can use you existing SQL skills to get going with analysis of the type and volume of semi-structured data that would be awkward for a relational database. With an IDE such as Databricks you can very quickly get hands-on experience with an interesting technology.… Read more
09 November 2015
09 November 2015

The Logical Data Warehouse – Towards a Single View of All the Data

0
15
What is wrong with the Enterprise Data Warehouse? Quite a lot, it seems. By taking the narrow view that the struggle is that of accommodating and interrogating huge quantities of data, then initiatives such as the Virtual Data Warehouse and Logical Data Warehouse could make sense. But what about data quality, security, access control, archiving, retention, privacy and regulatory compliance?… Read more
10 March 2015
10 March 2015

The Internet of Things: A New World Order?

0
19
Was the marketing hook 'The Internet of Things' conjured up before the technical definition? Are we being persuaded to spend money on fending off yet another fantasy tsunami of data? Already, we have televisions that listen to, and report, your conversations; so are we facing the Science Fiction future of gadgets that report where you go, who you visit and what medications you take? As Robert Sheldon says; "It's big, almost too big to get your arms around"… Read more
21 May 2013
21 May 2013

Big Data: Are We Looking at the Wrong “V”s?

0
14
For once, business people are excited about the importance of data and are interested in the business benefits of extracting insights from the data. Perhaps this is more of a cultural rather a technical initiative, and so we in the data industry should participate by redefining the concept of the three Vs to reflect business values rather than technical challenges.… Read more
05 September 2012
05 September 2012

Analyze Big Data with Apache Hadoop on Windows Azure Preview Service Update 3

Hadoop and MapReduce have good prospects for adoption as a standard for big data analysis, especially since its adoption by Microsoft. It is ideal for Cloud usage since one can spin up nodes when required, pay only for storage and compute services whilst they are running. Roger Jennings describes how to get it running on Azure.… Read more