{"id":73300,"date":"2012-07-16T16:40:42","date_gmt":"2012-07-16T16:40:42","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/uncategorized\/hate-the-player-not-the-game-or-my-case-for-data-engineers\/"},"modified":"2021-07-14T13:07:54","modified_gmt":"2021-07-14T13:07:54","slug":"hate-the-player-not-the-game-or-my-case-for-data-engineers","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/databases\/oracle-databases\/hate-the-player-not-the-game-or-my-case-for-data-engineers\/","title":{"rendered":"Hate the player, not the game \u2014 or my case for Data Engineers"},"content":{"rendered":"

Has \u201cdatabase\u201d become a dirty word within your organization lately? If you\u2019re someone who has been a data technologist for the better part of your career, you may be wondering why the technologies you work with everyday seem to be acquiring such a bad rap. From NoSQL to\u00a0No DB<\/a>\u00a0the current influx of brogrammers seem to take extreme pride in describing how they\u2019re able to write code while avoiding any kind of database technology whatsoever.<\/p>\n

The impetus for this post actually started with something I read on the \u2018Net the other day about Command Query Responsibility Segregation (CQRS), and how I was initially excited about the concept.<\/p>\n

Martin Fowler has a nice, gentle introduction the topic\u00a0here<\/a>.<\/p>\n

Before I get into the post, however, I think it\u2019s useful for me to describe some of my attitudes toward data management. What\u2019s really odd is that while I rather strongly disagree with the tone of\u00a0Uncle Bob Martin\u2019s Rant<\/a>, I actually strongly agree with his assertion about the high value of use-case driven development.<\/p>\n

I\u2019ve had gentle debates about the meaning of \u201cdata as truth\u201d with several people, and the age-old debate of whether data is more \u201cimportant\u201d than application code. Generally I\u2019ve found that such debates end up as religious arguments instead of attempting to get to the value of acting on data \/ or data in action. Because in the end it\u2019s hard for data to have value unless its acted on by a set of processing directives (applications), and while it\u2019s possible to have valuable applications that don\u2019t require knowledge about the past (basic rule engine codifications), in general they need each other.<\/p>\n

Why I call myself a data engineer<\/p>\n

I\u2019ve been impressed with EMC\u2019s attempt to define a Data Science curriculum. In particular, I like how they describe the different skills and roles necessary for a successful data science team, including the hot new title of data scientist. The data science team often includes a data architect, a data engineer, and a database administrator. So, what is a data engineer? In a\u00a0blog by Steve Todd<\/a>, Director of EMC\u2019s Global Research and Innovation Portfolio, he has the following characterizations:<\/p>\n

\n

The \u201cDatabase Administrator\u201d provisions and configures the database environment to support the analytical needs of the working team. The \u201cData Engineer\u201d tends to have deep technical skills to assist with tuning SQL queries for data management and extraction. They also support data ingest to the analytic sandbox. These people can be one in the same, but many times the data engineer is an expert on queries and data manipulation (and not necessarily analytics as such). The DBA may be good at this too, but many times they may simply be someone who is primarily skilled at setting up and deploying a large database schema, or product, or stack.<\/p>\n<\/blockquote>\n

Many, many DBAs wear both hats, but I think it\u2019s not a good idea \u2014 in general I think that DBA is to data engineer as system administrator is to software engineer, but the lack of data engineers has forced DBAs into dual-roles, often for which they are not well-suited. While I have basic DBA skills, I\u2019m much better at the skills listed under the data engineer \u2014 and I enjoy working with the data scientists or application developers who have questions about the data and\/or how they\u2019d like it structured to support their use cases.<\/p>\n

This is one of the reasons why I agree with Uncle Bob\u2019s rant in which he also rails against frameworks in addition to the database \u2014 I just wish frameworks had received equal billing in the rant and title, but I\u2019m guessing that the No DB vitriol resonated more highly with readers. In general I like making sure data is organized in such a way as to support as many use cases as possible. That includes being performant for each use case \u2014 which may mean taking advantage of techniques to denormalize, duplicate and synchronize, cache and distribute data.<\/p>\n

Continue reading…<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Has \u201cdatabase\u201d become a dirty word within your organization lately? If you\u2019re someone who has been a data technologist for the better part of your career, you may be wondering why the technologies you work with everyday seem to be acquiring such a bad rap. From NoSQL to\u00a0No DB\u00a0the current influx of brogrammers seem to take extreme pride in describing…<\/p>\n","protected":false},"author":316178,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[143533],"tags":[],"coauthors":[],"class_list":["post-73300","post","type-post","status-publish","format-standard","hentry","category-oracle-databases"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/73300","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/316178"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=73300"}],"version-history":[{"count":1,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/73300\/revisions"}],"predecessor-version":[{"id":91752,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/73300\/revisions\/91752"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=73300"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=73300"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=73300"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=73300"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}