Towards the end of the summer of 1969 – a few weeks after the Apollo 11 moon landings, a large grey metal box was delivered to the office of Leonard Kleinrock, a professor at the University of California in Los Angeles. It was the same size and shape as a household refrigerator, and outwardly, at least, it had about as much charm. It was an IMP, or ‘interface message processor’ the first networking hardware of the internet.

It’s impossible to say for certain when the internet (or Arpanet as it was then called) came into existence but 29 October 1969 has a strong claim for being ‘the day the infant internet uttered its first words’.

“I know where the
dangerous waters lie
and I know how to run
my system ‘properly’
but most users don’t,
and don’t understand how
their behaviour opens
their systems to problems”…

At 10.30pm on that day, as Kleinrock’s fellow professors and students crowded around, a computer was connected to the IMP, which made contact with a second IMP, attached to a second computer, several hundred miles away at the Stanford Research Institute, and an undergraduate named Charley Kline tapped out a message.

Samuel Morse, sending the first telegraph message 125 years previously, chose the portentous phrase: ‘What hath God wrought?’.

But Kline’s task was to log in remotely from LA to the Stanford machine, and there was no opportunity for portentousness: his instructions were to type the command LOGIN.

The IMPS had been designed and built by Bolt, Beranek and Newman (BBN) and had been written a team of trio programmers. Berbie Cosell was one of the three. Over a career that spanned 26 years at BBN, Cosell earned a reputation as an outstanding debugger who could be thrown into any situation and make the software work. He says, rather modestly that he was just ‘ a very careful programmer.’

An old-style hacker he wrote DOCTOR, a version of Weizenbaum’s ELIZA which had a wide distribution inspiring new implementations and related programs. Though he officially retired to a sheep farm (Fantasy Farm) in Virginia in 1991, with his wife Lynn and a menarie of animals, he continues to hack and teaches programming and computer security. Ironically, Cosell, one of the father’s of the internet, relies on dial-up access from his home.

RM:

What was the biggest obstacle with writing the IMP software?

BC:

Well, there were a lot of obstacles; I’m not sure which might have been ‘biggest’. I’d think that perhaps the biggest obstacle was dealing with the distributed aspects of the system. Almost every part of the system didn’t really do anything: it had to interact with another system over new hardware, with no underlying operating system [we were writing that, too, of course] and with protocols we were inventing. Figuring out what was wrong and why when something failed was fairly tricky.

RM:

Was there any real-time debugging being done?

BC:

For sure – that was the only kind of debugging we could really do. We had to fire up the system run through some test scenario and see what happened and then figure out what broke.

RM:

How did you convince the other programmers on the IMP project to stop patching the binary?

BC:

Dave Walden was OK with reassembling, but Will Crowther wasn’t: Will was convinced that for every bug you fixed in editing/reassembling you introduced two new ones. I was pretty adamant about it, so I mostly did it by brute force: Basically, when we packed up in the afternoon, I’d collect the listings and notes we’d made during the day and when I got home I’d edit and reassemble, print out new assembly-listings and punch out new binary tapes. Whoever got in first would collect the new stuff and load it up and put the listings into binders and away we’d go with the new, no patches needed, version. So it wasn’t so much ‘convince’ as ‘just do it’ and when it worked out well there wasn’t really any resistance to continuing – especially since I was doing almost all of the work.

RM:

What was your strategy? Do you think de-bugging is intuitive or can it be taught?

BC:

For getting the IMP debugged, we mostly worked through the system piece by piece. We got the system ‘idling’ fairly quickly and we would then run test scenarios and see what happened (for example, connect two together and see if they did the handshake protocol and brought the line up; if so, we moved on, if not we tried to figure out why not). After the line was up we’d see if the routing got updated properly, and then try sending data over the line, etc. Just step-by-step testing all the functionality of the system.

As for the second, I really don’t know. I know that different people debug in different ways and it really does feel to me more like ‘art’ than ‘engineering’. What I would teach is the idea that you have to know what the program is supposed to do and that when you see an anomaly you need to try to understand why it is there and how it could have arisen – that computers and their software are basically deterministic and so everything that goes wrong is due to something being broken and you need to find it and fix it.

RM:

It seems extraordinary now, but way back in the early days of ARPANET few people could see the benefits of getting involved in networking or timesharing. Were you one of those sceptics?

BC:

No. I never would have guessed it would become what it did, but it was obvious to me that timesharing was certainly the ‘right idea’ and networking seemed like a potentially pretty useful technology – although I doubt any of us could have guessed what it would blossom into and the changes it would bring.

Prior to the ARPAnet IMP software I had worked on the Hospital timesharing system on our PDP-1 and that certainly convinced me that timesharing was a good thing. And as we brought the ARPAnet to life, it was fairly clear (to us, as the engineers) that it was going to work – we couldn’t yet see what it would really be ­good for, but it was such good technology that it was sure to be useful. There wasn’t a ‘killer app’ yet for the ARPAnet (that wouldn’t come until 1972 when Ray Tomlinson sent email over the ARPAnet) but it was already apparent that it was going to be very useful. As I say, it wasn’t clear to us that we were planting the engineering-seeds of something that would eventually change the world, but it was clear that it was going to survive and be useful.

RM:

What advice do you have for the large number of programmers who are self-taught?

BC:

Write lots of programs. Find things that interest you and write programs relating to that – so you’ll be motivated and interested in the pursuits. The more programs you write, the more you’ll be able to internalize how computers and systems work.

Also, while you’re doing that you should pay attention to the code you’re writing. If you’re having a hard time getting something to work, don’t just hit it with a sledge hammer until you brute-force get it working, but ­think about ­why the program is having trouble: what did you overlook in the design and implementation? What didn’t you think through clearly enough?

And also, remember that computers do what they’re told and so that cuts both ways: on the one hand, you need to completely understand what it is you want the program to do, and how to do it, and on the other hand if the program does anything wrong, or even suspicious, don’t just shrug it off as being ‘good enough’: go in and figure out what’s awry and make it better.

RM:

When I spoke to Niklaus Wirth I asked him about Open Source versus proprietary software and he said remained sceptical about Open Source because the systems available are complex, and who wanted to read and comprehend millions of lines of code before using these programs? What do you think of that?

BC:

I think that’s a bit too pessimistic. If you want to work on the program you’ll need to absorb a lot of information – programs are, indeed, very complicated beasts these days but that doesn’t really affect the open/closed matter much, other than for some programs the only way you can get to work on them is to get hired by the right company.

However, you don’t need to do that just to ­use the program. When you download sendmail or purchase PhotoShop, as a user it really makes little difference that the program is open or closed source: your concern is “does it work or not?”

If anything, open source makes it easier to be able to trust new and complicated programs and systems, since other engineers can poke through and evaluate the software and you can take advantage of independent reviews of the system. You don’t need to understand the gcc C-code optimizer (much less read through the code to see how it actually works) to be willing to use it to compile your C programs. I guess what I’m saying is that in the large it isn’t necessary that you read and comprehend the program but that because someone can and that possibility/reality there are more potential sources of tech support and also more eyes double-checking the code.

RM:

What do you think the answer is to the grumbling about the complexity, the size and lack of reliability of widely used software systems for PCs and servers? Would opening them to the designer from day one help matters?

BC:

The complexity is unavoidable because each generation builds on what came before and so the complexity is inherent and piles up. And, of course, users then expect applications to continue to build on what came before and so complexity begets more complexity.

Each generation of software engineer inherits all of the existing systems, libraries, etc, from the previous generation and builds on it. This has several implications. First is that a given generation of engineer need not necessarily understand how the underlying layers they’re building on actually work – they can just assume that those layers do what they’re expected to do. But this has been the case for decades: few people knew how the innards of an operating system like Multics or OS/360 or VMS worked but that didn’t stop anyone from using those systems. And as hardware and software have matured user expectations – and lack of interest in the gory details – have kept pace.

Second is that there’s no real reason for an engineer to spend time digging into the earlier ‘layers’ – the existing software environment (operating systems, protocol stacks, programming languages, utility applications) are all in place and can just be taken as-is. I have ­no idea how MySQL does all the magic it does, but that doesn’t stop me from building DB-based applications that expect MySQL to handle the database. I have no idea how browsers manage to actually render and display HTML, but I build CGI-based applications that assume that the browser will do what it needs to.

The reliability problems arise in part, I think, because of this complexity. When systems could still largely be comprehended by one person, that engineer could try to ensure that the whole system made sense, fit together, etc. But when systems are developed as layer upon layer of independently implemented packages, it gets very hard for there not to be ‘cracks’ between the layers.

And for applications, they have become very complicated. In fact, so complicated that it strikes me as not all that surprising that bugs can creep in. It is almost hard to completely test the basic functionality of these utilities much less every internal interaction. This was always a problem for me: I was just not creative enough in thinking of ‘dumb’ things to try and so would miss a subtle lurking bug when a user did something really bizarre. Like ‘who would think to test send a line printer listing to the paper tape reader’ and the like.

But I don’t see much of that ‘lack of reliability’. I think that modern software is not much short of remarkable. I know a bit about what it has to be doing ‘under the hood’ to do what it does and I’m often stunned at just how well, and how reliably, it does it.

I think that some of the bad-rap some software and systems get is due to a human-factors/human-interface problem. The average user basically expects their computer to be as simple and predictable as a toaster or CD player, when the system is actually an almost unfathomably complicated bit of engineering. For someone like me, I have little trouble: I know where the dangerous waters lie and I know how to run my system ‘properly’ but most users don’t and don’t understand how their behaviour opens their systems to problems. Just in the last few years MS and Apple have advanced a huge amount in dealing with this aspect: making the system, as complicated and powerful as it is, more idiot-proof.

RM:

Has the nature of programming changed as a consequence of so many companies cherishing complexity as an effective prevention against their work being copied?

BC:

I can’t comment on that motivation for complexity, but the nature of programming has definitely changed. Modern programming is much more abstract than it used to be. The operating systems and support libraries and hardware are so mature and so stable that there’s hardly any need for an application programmer to worry about details of how the unpinnings work.  There’s hardly any need for a programmer to delve into how an RDBMS actually parses and executes a SQL statement or what goes on in the inner workings of Perl.

As for being copied, I don’t think that ‘cherishing complexity’ is much of an impediment. I have believed for a long time that there’s a very strong ‘second system’ effect. That doing something first can be very hard, but once you see something that works and see that it is successful in the marketplace, doing it second is almost always comparatively easy.

RM:

The technology industry has developed out of all recognition since you began your career, what do you think are the good things about this sea-change and what are the bad?

BC:

The good thing is that programmers can now write programs that we couldn’t have dreamt of back then. When they add to the intellectual mountain, they get to build wonderful things on top of the decades of superstructure and so they can reach new heights farther, faster and easier than we could ever have done.

Well, one downside is that there’s less intuition about what all the pieces are and how they fit together, but I’m not sure that’s all that big a downside.. I no longer understand how my car works, but that doesn’t bother me and my car works better and more reliably than my cars did back decades ago when I did still understand how to adjust the carburettor and set the points.

Perhaps another slight downside is that it is harder to get an ‘in’ into the industry. The tools and systems are so good that folks with minimal training and experience can do a passable job, and so it is tricky for a really good engineer without much experience to stand out – they’ll be better than the pack, but the floor has been raised so high that they won’t be obvious stars.

I talked with some Virginia Tech students a few years back and they were a bit despondent at how few offers of good jobs their proper-computer-science degrees were netting them in competition with people with just a year or two at one of those ‘technical institutes’ and so they were asking me how do they get to a place where they can really show how good they are, and I had to admit that I didn’t (and don’t) really know.

When things were more primitive and almost nobody knew anything, employers would take a chance on someone like me (when BBN hired me to join the hospital project I had precisely zero professional programming experience, and close to none informal programming experience, but in a few months they had promoted me to being the lead systems programmer for the PDP-1 timesharing system. I can’t imagine something like that happening these days.