No, things do not need to be more simple – they need to be more complicated

“I despair. Technology, and our very civilization, will get more and more complex until it collapses. There is no opposing pressure to limit this growth.” Chuck Moore 2009

I recently sat through a meeting at a company where a Chief Information Officer described what he wanted from his Information Technology Department’s latest project, which was no less than a significant re-architecture of their primary revenue-generating system. As he pontificated along, he made a statement that I’ve heard several times in my career from a senior leader within the industry: “This just feels too complicated. We have to make it simple!”

You’ve probably heard something similar in your tenure in IT. As we consider the myriad options to accomplish what we need to do, we’re sometimes overwhelmed with what seems to be an over-complicated set of processes and tools to get something done. It just feels like it should be simpler or easier to do.

And yet, I believe that the solutions we implement should be more complicated. That’s right; we should not fear complication, but instead seek it out.

This may seem counter-intuitive and therefore wrong, but here we begin a journey into both the English language, technology and psychology. This is a technical discussion, however, all driven by a decision about using the “Cloud”, or more accurately, distributed computing.The discussion of simple versus complicated as it pertains to technology is rather complicated.

As Socrates was said to have explained, “The definition of terms is the beginning of wisdom”, so I’ll start with setting out a few of the terms I’ll use throughout this discussion. I’m rather fond of using the Oxford English Dictionary (OED), so as l began thinking about this topic, I used that to lookup these words:

• SIMPLE: adjective; 1) Easily understood or done; presenting no difficulty; plain, basic, or uncomplicated inform, nature, or design; without much decoration or ornamentation 2) composed of a single element; not compound.
• COMPLICATED: adjective; Consisting of an intimate combination of parts or elements not easy to unravel or separate; involved, intricate, confused.
• COMPLEX: adjective; 1) Consisting of or comprehending various parts united or connected together; formed by combination of different elements; composite, compound. Said of things, ideas, etc. 2) Especially, consisting of parts or elements not simply co-ordinated, but some of them involved in various degrees of subordination; complicated, involved, intricate; not easily analysed or disentangled.

This is more than an academic exercise. It is essential to the point I’m making  to get  the meaning of these terms out in the open. In particular, I found a rather lively debate on the word-nerd sites about the difference (or lack thereof) between the words ‘complicated’ and ‘complex’. Whilst most people use the words interchangeably, there does seem to be a subtle difference. Something that is complex involves multiple parts, some of which are not completely related. Something that is complicated involves a level of time and understanding, or more accurately, time to understand. And that difference forms the crux of my argument.

Simple is, well, fairly simple to understand. Interestingly, in it’s description also lies the crux of my argument – it involves few, or even just one, element or process. As anyone in technology will tell you, nothing we do is simple by the strict definition of that word.

 Lest you think I’m making a purely syntactic argument, l do think l understand what our erstwhile CEO was attempting to convey. He looks at the single-button, glass-screened design of his Smartphone where he slides a finger and selects a glossy picture of a movie reel (many movies are digital now anyway, so this too is an anachronism) to buy a set of tickets to the evening’s entertainment Within three clicks he has the tickets waiting in cyberspace for him to show the QR-Code at the door of the theater- simple! This is the simplicity he seeks – but it is misplaced.

 In this particular discussion, I was part of a panel of architects assisting the company. As we began to explain the various options he could use to re-architect his company’s system (which, as you recall, is quite non-trivial), his eyes began to glaze, he focused on his phone and made the famous statement. And I understand what he wanted. Isn’t there just a box of software somewhere that will let me do what I want to do without all the complexity of Infrastructure as a Service, Platform as a Service, Software as a Service, authorization and authentication choices, storage options, and hundreds of other points?

No – actually, there isn’t.

Lure of the simple

And here is where we begin the journey from the English language to the overwhelming human psychological need for control. Not just physical control and manipulation, but actually being able to fit it all into a set of concepts that can be easily grasped. We need this: If we don’t understand something, it is inherently distrusted.

The primary reason for this is that trust factor. When I know the details of something, mentally I don’t fear it as much as I do when I don’t understand something. To be sure, I may still respect it-such as in the case of electricity, fire and so on – but I don’t fear it. When a manager or client sees all of the moving parts, processes and people in a solution they begin to become overwhelmed by trying to fit it into simpler, larger concepts. And it begins to “feel” wrong. That feeling is mistrust. I see this quite often in my capacity as a vendor-since I represent a product, it is assumed that I will recommend a more complicated, and of course more expensive solution using the products I represent.

Fewer parts means less expensive, easier to understand, and more resilient and less susceptible to failure. That’s just common sense. We’re inundated with what appear to be simple products. However, what appears simple is often not, and simple does not always imply the values we ascribe to it.

But the error in seeking simplicity-at least on the back-end of a system-is that we risk using a less-optimal architecture than we should.

 Inescapability of the complex

I once sat in a meeting with a Business Analyst in a design review for a large system we were deploying in our enterprise. After a series of discussions around the room for he system, our CTO shook her head and said “this seems very complicated. Shouldn’t this be simpler?” The Business Analyst, one of the most calm, rational people I have ever met, looked at her and said “Some things are complicated because they’re complicated.” And he was right.

Implicit within new, more powerful tools, equipment and technology is increased complication. Think of someone living 100 years ago. Communication options were few-speech, the written word (on paper), telegraphs and for the privileged few, phones and other electronic devices. Compare that to the dizzying array of methods that we have today. Even the “smart” phones we consider simple can be baffling to someone who has never used one. How do you turn it on? What’s a “lock code”? Where is the dial-tone? What are “bars”?To modern users, simple. To someone not immersed in the culture and following along with the incremental changes in technology, quite difficult – and not simple.

 When I first installed Windows 8 preview on my laptop, I was able to use the Metro interface quite quickly. But a friend noticed that I made several movements to show various programs. She chuckled and said “You know you can just start typing at the main screen, correct? It will narrow the options as you type.” What could be simpler? A single action (start typing) was complicated because at the first I didn’t understand that it was legitimate to take that action. I had been trained to look for a button, icon or some other indication of what l was supposed to do. I never considered that the interface was right under my fingers.

It’s obvious that a jet aircraft is more powerful than a car – and that it is more complicated as well. Ditto for the car as opposed to a bicycle, and a bicycle to a pair of shoes. But even though we might not completely understand all of the physics, components and parts within the jet aircraft we trust it. Or rather, we trust that the system that is built around the jet is designed to harness that complexity to give us certainty about the process of flying.

And so it goes with computing technology. Each new advancement brings with it new knowledge we have to learn, synthesize and implement. Things don’t become more simple-they become more complicated. And that is not only to be expected, but in fact embraced. Especially within the technology field, we need to be grounded in the basics but willing to discard hard-won knowledge for new concepts and processes. It’s our challenge to ensure the organization we serve has the best of the newest technology. That doesn’t imply we have to adopt every new paradigm, of course. But it does mean we need to learn new complexities to evaluate the proper use of a technology to accomplish the organization’s goals.

 The pursuit of the simple at the expense of the right solution is not a strategic, or even tactical advantage. It’s a limitation-one you should not fall prey to.

 So if you shouldn’t seek out the simple, what should you do? You should not fear the complexity of a  solution as you design it. It’s a matter of breaking down the complicated to the disparate parts, understanding the options you have, and selecting the ones that fit the goals of the system. That might become quite complicated, which is completely acceptable. Document, design, and always compare a particular component of your solution to the goal or goals – if they don’t match up, start with a new component.

This means you have a lot of things to learn. You need to become adept at understanding new concepts and being able to jettison old ones, which can be irritating. “I spent a long time learning X, and now they come out with Y!” Not only are both of those statements true, they will be true again, probably in 6 months to a year later. It’s a cycle we have to get comfortable with.

 But just as eating is important, and over-eating is dangerous, I make a distinction here against over- complication. I’m not suggesting that you complicate things that don’t need to be complicated. My process for designing an architecture is to understand the problem (or thesis) as completely as time allows. Then I research and document the options for the components in the problems. I route the problems through the options based on the variables-and then I optimize. I work from the general to the specific, creating categories or “buckets” and working to make those as efficient and effective as possible. I don’t worry about how complicated -or how complex -the solution is.

Simple should follow complicated

So it would seem that my position is that you ignore a goal of simplicity, but this is incorrect. In fact, we often think that simple precedes complicated. But in fact, it’s the opposite. The smart phone is quite simple, once you get a few key concepts down. But making that mobile computing device is anything but simple. It’s actually quite difficult-both complicated and complex-to make something simple to use.

And this is where the crux of the simplicity argument fails. The CEO in question misplaced where the simplicity lives. Sure, for the end user, the solution should be as simple as possible. But the implementation of that solution is often wildly complicated. Yes, that means there are more places for the solution to fail. There is more to know, to manage, and to understand. This is especially true as we move into (actually return to) the age of distributed computing, or the cloud. We’re adding complexity by adding new ways of doing things. That isn’t something to fear, but something to understand and apply properly where it fits.

I rarely suggest that a company move an existing, functioning application into the cloud, but instead to extend or work in a hybrid fashion to solve a problem where it fits. And as a technology matures, it gets simpler- at least somewhat. When I started writing software, I worked in Assembly. This was quite complex, and not at all English-like. From there newer languages were developed that moved higher up the stack so that I can now write powerful, graphical programs in relative ease. But make no mistake – much of this is because instruction sets for the program have moved into the language I am using, or even all the way down into the CISC chips I am using on the computer. The complexity isn’t necessarily gone; it’s just hidden from me. It’s still buried within the system. Even so, that makes it easier for me, and I’m fine with that.

So as you design a solution, you should certainly seek to make the system simpler-but not at the expense of correctness. Roger Sessions, paraphrasing a much longer statement by Albert Einstein, said “Everything should be as simple as possible, but no simpler.” This is another way to state Ockham’s Razor: “one should proceed to simpler theories to the point at which further simplicity can only be achieved at the expense of explanatory power”.

 I’ll leave you with a final thought. When I heard the CEO utter this statement, I knew my task was clear. I needed to understand what he was looking for-a conceptually clearer, easier to understand solution, that he could quickly grasp. I took the solution we were proposing, moved the components into larger conceptual areas, and encapsulated the ideas they held into fewer steps. This “simplicity” hid the details without deception, and met the requirements of the solution. One of the most important skilIs that you can learn is to synthesize your ideas into ever-increasing levels of detail, starting with the most general.