The Data Dialog

Most companies have long since jettisoned the role of the grizzled old data architect, whose job it was to ensure a common understanding of the meaning of all data entities in the enterprise, and the actions carried out on it. Many are just now coming to realize what a costly mistake that might have been...

Boss: Phil, this strategy paper that you’ve written on our corporate data integration policy. Look, I’m the IT director and even for me it’s pretty opaque. Damned difficult to read, in fact!
Phil [blushing modestly]: Why, thank you. I try to do my best.
Boss: But damn it, Phil, we can’t present this to the board for signoff! What is it actually trying to say?
Phil: Well, in essence, it says that our new data systems will cost twice as much as we predicted, and take twice as long to implement. And, by the way, our last project that promised to take an integrated approach to the problem of corporate data management crashed and burned without trace.
Boss: What!? I never read about any of that!
Phil: [pointing]: It’s right there on page 3, tucked into that long paragraph near the end…
Boss: Damn it again, Phil. How often do I have to tell you that we senior staff only like to read the first page of these documents?
Phil: Precisely! We just need to make sure they can’t claim that they were never informed about the outcome of our IT initiatives. We don’t want them to actually read it…that would be disastrous.
Boss: …and what about all these terms in the document…MDM, BI, CDI, PIM, ERP, ETL, EAI, EII, EIS, ST, SOA and SDM?
Phil: They are all examples of IDC (Initials Designed to Confuse), also known as ADH (Acronyms Designed to Humiliate). They are very useful ammunition against the Geeks when they try the same tactics with their .NET and J2EE jargon but, as is the case here, they are mainly used to browbeat Business managers with our knowledge and intellect.

Many IT initiatives are so complex as to be mystical quasi-religions. They are difficult to criticize because their proponents can always sorrowfully shake their heads and protest that you simply don’t understand enough about them.

Boss: [sighing wearily]: What about this part here…the analysis of the current system. What does this mean? [pointing]
Phil: [peering over his shoulder]:
‘Diverse system architectures‘ means that the bloody computer systems can’t communicate; ‘diverse business functions‘ means that even the business can’t communicate; a ‘meta-data abstraction layer‘ is the database equivalent of Harvester Tape. Referring to a ‘consistent information framework‘ means that at least the column names in the databases are the same and mean the same thing. Where I refer to people as ‘stakeholders‘, it means I don’t know what the heck they contribute to the business, or can’t remember the right word for it. If an IT system ‘gives micro and macro business process capabilities‘, it means that it ‘does stuff’
Boss: You are going at it pretty strong on the past failings and mistakes of IT! Aren’t you overdoing it?
Phil: Curiously, any business is only too ready to believe that IT makes mistakes; I sprinkle in these confessions to give the whole document verisimilitude.
Boss: OK but why then leave out the two biggest mistakes of all…the Business Re-Engineering project and the Data-Warehousing project?
Phil: Because the business management still remember how much they paid IT to implement them. It may have a negative impact if we confess they were blind alleys. Also, if we mention those, they may also spot that this new MDM (Master Data Management) strategy is just a re-arrangement of the furniture to sell the same old apartment.
Boss: So what is the purpose of this document? It doesn’t provide information.
Phil: [shocked]: Thank goodness! IT business documents aren’t there to provide “information”! They are there to provide cover and concealment for our real activities
Boss: Come on Phil, don’t be so silly, and just rephrase the entire document so that ordinary mortals like me can understand it. Tell me what it means as though I was a chum at the golf club
Phil: [apprehensively]: Are you absolutely sure you want that?
Boss: Of course, get on with it man!
Phil: [gulping nervously]:

OK…

Once upon a time, IT departments used to appoint someone to keep tabs on what data was held by the company, what the business understood about that data and the sort of processes that happened to that data. This person, usually a grizzled Systems Analyst (or Data Architect), would ensure that there was a common understanding of all the real entities involved, such as customers, products, employees, and the processes that acted upon them, such as releases, recalls, invoices, and so on. He would maintain a document, usually called a Data Dictionary or Data Model, which could be read and understood by anyone, and provided a basis for each individual IT project. Any misunderstandings were soon ironed out, and the activity was an essential part of strategic planning.

When the cyclic downturns in IT happened, it became tempting to get rid of this chore. The Data Architect, or whatever he was called, was packed off with an ornamental mantel clock under his arm, and there seemed to be no dire consequences. It all seemed painless. The integrity, and shared understanding, of Corporate data didn’t seem to be an important asset after all.

However, with mergers, acquisitions and restructurings, many enterprises, including our own, have struggled to keep a consistent culture and nomenclature. The first sign of the disease is the increasing number of anomalies that seem to slip in to the company reports. Often, the root cause of such anomalies is laughably crude, such as the case when one division of a major bank counted a customer as a person who had one or more accounts, whereas another counted a customer as an account-holder.

The problems quickly became endemic and the only solution was to perform a major business-re-engineering. This suited the hotheads, but led to several high-profile disasters. The idea of ‘Data Warehousing’ was an attractive panacea, as it meant that nobody had to confront the essential problem of ‘data-confusion’ within the business. One could fiddle with the messy data and pummel it into a logical state, from which one could gather all that vital business information and reporting. Most bought into the dream.

Many a silver-haired ex-Data Architect will have smiled as he sat in his deck-chair, reading about the vast cost and wasted effort involved in implementing a dream that couldn’t live up to its promise. If data is fundamentally unsound, no manipulation can help it. As they say in Essex, ‘you can’t turn turds into plum pudding’.

With this company’s adoption of distributed architectures and complex messaging, the opportunities for confusion have increased. Whereas, one once had just to keep a consensus on what business entities comprised at the data layer, now one has to browbeat anyone who has the wit to nail an XML message together.

So we now come to the point where what this paper is suggesting is a type of Master Data Management initiative. We once more realise the importance of understanding the ‘corporate metadata’, the consensus on how the corporate entities are understood and accounted, and the nature of the processes that act on them. We understand a service-based approach to data and we now have the technology to integrate this model into our IT systems. And, ironically, it is only now that we have come to realize the importance of that ancient Systems analyst poring over his obscure diagrams of boxes and arrows. His prolonged absence is the reason why this new data system will now cost so much, and take so long to implement. And we still might not get it right…

Boss: Hmm. On second thoughts, I doubt if the business is really ready for the truth. Perhaps we’ll go back to your original three-pager.
Phil: [relieved] I think it will have the effect that we want.