Donald Knuth: Geek of the Week

Donald Knuth is an extraordinary man. As well as inventing 'Literate Programming' and writing the most important textbook on programming algorithms, he is also famous for designing and programming one of the most widely-used digital typesetting systems ever, even designing the fonts that went with it. He also pioneered the use of 'Open-source' software. Knuth is a man of engaging charm and enthusiasms who combines a knowledge of history, music, art and mathematics with a unique insight into the art of computer programming.

It is often said that entrepreneurial companies tend to reflect the personality of their founder, and nowhere was this more true than at Microsoft. Bill Gates was by turns very brilliant, very arrogant, a master marketer, a win-at-any-cost competitor – and so was his company.

873-knuth_don.jpg

A lot of people have been in awe of Gates but equally there are people Gates lionises and one of these is the legendary American mathematician and computer scientist Donald Knuth. Gates once said of Knuth’s most famous book The Art of Computer Programming ‘If you think you’re a really good programmer . . . read (Knuth’s) Art of Computer Programming . . . You should definitely send me a resume if you can read the whole thing’

But of all the plaudits rained on him perhaps the most outstanding happened in the late 1990s when The Art of Computer Programming was named among the best twelve physical-science monographs of the century by American Scientist magazine, along with Dirac on quantum mechanics and Einstein on relativity.

Knuth has always viewed the stages of writing The Art of Computing Programming, which was originally conceived as a single book, in 1962, as the most important project of his life. He famously eschews email telling Google co-founder Sergey Brin, a former student of his that:

‘E-mail is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration.’

He says that his full-time writing schedule means that he is ‘pretty much a hermit… concentrating intensively and uninterruptedly on one subject at a time, rather than swapping a number of topics in and out of my head. I’m unable to schedule appointments with visitors, travel to conferences or accept speaking engagements, or undertake any new responsibilities of any kind.’

In the forty-seven years since beginning the book that has almost defined computer programming as much as it has defined him, Knuth has received awards including the Kyoto Prize (1996), the Turing Award (1974), and the National Medal of Science (1979). The irony is that computer science nearly lost Knuth to its ranks because of his love of music (his house is built around a two-storey pipe organ that he designed himself) and says he intends to return to it once he has completed the expected seven volumes of ‘The Art of Computer Science’.

“People these days rarely
measure a computer
scientist by standards
of beauty and interest;
they measure us by dollars”

Born in Milwaukee Wisconsin he chose physics over music as his major subject at the Case Institute of Technology. At Case, he managed the basketball team and applied his talents by constructing a formula for the value of each player. He then went on in 1963 to earn a Ph.D. in mathematics from the California Institute of Technology.

A pioneer in computer science, he took time out during the 1970s from writing ‘The Art’ in order to develop TeX, a document-preparation system which has become a standard for submitting typeset-ready scientific and mathematical research papers for publication. A prolific writer he has also written a science-fiction novel, Surreal Numbers: How Two Ex-Students Turned On to Pure Mathematics and Found Total Happiness (1974), based on a number system invented by the mathematician John Conway.

RM:
When you published the first volume of The Art of Computer Programming did you consider yourself a spokesman for a new generation of computer scientists? Did you have the feeling that this book was a seminal work?
DK:
I recall considering myself as a spokesman for the ‘old generation of computer scientists’, who mostly hadn’t written up their great ideas from a neutral standpoint; they had tended to describe only their own points of view.
I figured that I could present their ideas in a more balanced way, having no personal axes to grind because I hadn’t discovered very many things by myself. I knew that there was a great story waiting to be told, and several dozen people had been encouraging me to write it down for a new generation of students who needed texts from which to study.
RM:
Your work on The Art of Computer Programming interests me most obviously but I’d like to ask you about your writing. Do you revise endlessly and how do you know when something is right?
DK:
In high school and college I had done a fair amount of writing, so I’d had a lot of practice with exposition. My method has always been to dive in and write a first draft, then throw it out and start over once I see what story I really want to tell. I write first by longhand, with lots of erasing and crossing out. Then I type it up, editing for style, rhythm and tempo, then I look at it a while later and try to make it read right for a person who hasn’t seen it before. I try to avoid jargon unless it’s really necessary. I try to keep sentences upbeat, and to convey the sense of discovery that I had when learning the material.
I try to be precise and to say everything twice or thrice in complementary ways so that readers can absorb the material in two or three parts of their brain for ready retrieval later. I don’t emphasize cookbook techniques or buzzwords; rather I try to focus on the gist of ideas that I think will help to solve tomorrow’s problems. Of course I don’t always succeed; but I strive to be a teacher who stretches the readers’ minds.
RM:
What gets you started writing? Is it an idea, an image, a situation or event, a phrase, something else? Do you a particular reader in mind when you sit down and begin to compose or write?
DK:
Usually I try to start by recalling what things were key when I first learned a subject. The most important thing is to have good examples on which to hang the abstract ideas. If the example takes too long to explain or is too much tied to a particular application, I might lose the interest or attention of too many readers. Thus I spend a great deal of time trying to find examples that are simultaneously easy to grasp and quite instructive. Let me try to give examples of such examples: I once spent more than a month, perhaps two months, working out an elaborate example based on the game of baseball. I wrote lengthy computer programs to check this example; I designed a dozen fun exercises to walk the reader through it.
I had a big chart that was going to become a foldout illustration. I knew that baseball addicts would love me for this example. But finally I threw it out, and it will never see the light of day; I realized that it was mostly just fun for people who loved baseball, but it really didn’t teach much computer science.
And to somebody in Britain like you it would have been of about as much interest as an example based on cricket would be to me. Conversely, a year or so ago I was searching for an example of a technique that the experts call ‘solving 2SAT in linear time’; nothing in any papers I’d read had explained a nice reason why one would ever want to solve such a problem. I came up with an idea about scheduling comedians at hotels in Las Vegas, and voila: I could teach 2SAT and have fun at the same time.
Yes, I always try to keep a potential reader in mind – basically somebody who is reading my book because they want to, not because they must, and somebody who has a natural ability to do computer programming. Only about 2 per cent of the population really “resonates” with programming the way I do; but somebody ought to write books for that 2%, and I try to be one of the authors who does so. Instead of trying to impress the reader with what I know, I try to explain why the things I’ve learned impress me.
RM:
Does your emotional state have any bearing on your work?
DK:
I can’t write upbeat material when I’m feeling down. Mostly I guess I’m lucky to be basically happy, having a loving family and friends. I don’t use coffee or other stimulants to make me artificially alert; if I’m tired I take a nap. I need a quiet place to work, without interruption; that’s why I gave up email 20 years ago. Sometimes it’s hard to get started in the morning, when I think about how much still needs to be done; but once I get going on some technical problem, my juices start to flow and sometimes I’m so wound up that it’s hard to go to sleep at night.
That’s when I turn to light reading (Dorothy Sayers, Ian Fleming, Frederick Forsythe, and Agatha Christie, or Robert Barnard, say – you’ll see how much I like British authors – but also Raymond Chandler, Rex Stout, Herman Wouk, Sara Paretzky, etc.). A paragraph or two of such books often knocks me out so that I can sleep blissfully.
RM:
You have a special interest in music, particularly playing the organ. Can you listen to music, or be otherwise half-distracted when you’re working on something?
DK:
Different kinds of music work for me at different times. For instance, I found that I’m a much sharper proof-reader when listening to Telemann’s chamber music than when the room is quiet; conversely, if Bach is playing I can’t proofread at all. Some times when I’m working on the index to a book, Brahms symphonies are just the thing; I type in sync with the tympani. In the old days when I worked with a typewriter, I typed most of The Art of Computer Programming to the accompaniment of movies like Double Indemnity.
I probably watched that movie two dozen times (on the Late Late Show in Southern California) while working on Volume 1 during the 1960s; it has an excellent musical score, and a plot that’s just distracting enough to keep me from being bored while typing.
RM:
How extensive are the journals you have kept and do you hope to publish them?
DK:
I keep a technical diary that basically records what work I did each day, with a sentence or two about other activities. A few examples of these diaries appear in facsimile, in chapters 24 and 25 of my book Digital Typography. The diaries are on my computer, so that I can easily search for things that I did long ago; for example, if I want to know when I was in London, I can easily discover that I flew on 04 May 1965 and on 31 May 1967, 19 May 2009. I suppose these diary entries will be online some day, since there’s nothing secret about them. However they won’t be anywhere near as interesting as the diaries of, say, Lewis Carroll.
RM:
Do you think deep analysis of a problem or a solution to a problem is something that can be taught?
DK:
Yes I think it’s possible to learn ‘by osmosis’ from a good researcher how to do research. But, like programming, that seems to be something that certain people can do much more naturally than others.
I don’t believe that it’s possible to take any person off the street and teach them to do deep research, or make them into an expert programmer, any more than it would be possible to teach me how to be a great football player, By any definition of ‘football.’
When I was coaching grad students, I’d start with small problems. First a student solves a problem that was solved centuries ago, but the student did it independently. Then comes the solution to a problem that was first solved only a decade ago. Eventually, with the same research techniques, along comes a result that is totally new.
RM:
Would you still study computers whether or not it had any commercial value?
DK:
Thank you for asking that question. I have always been attracted to computer science because it involves beautiful patterns, rather like the way dancers enjoy choreography, and because questions such as “What can be computed efficiently?” are profoundly interesting and challenging.
Conversely, I’ve never understood why anybody pays for computers, nor do I try to create things that will make a fortune for me or anybody else.
As a teacher, I’ve always tried to communicate important information; and this gave me a decent salary. As a writer, I don’t give any thought to what books will sell best; I try to imagine which books are needed most, or which books would be most fun to read or write.
Fortunately I came along at a time when I could write books that people were willing to pay for, and that money has allowed me to buy a pipe organ and to support various charities.
If I were forced to use my programming skills to support myself, or if I considered myself a chief and leader more than as a member of a tribe, my attitude would no doubt be quite different. But I really would be a computer scientist even if the discipline were completely detached from the economic scene.
Therefore I am disappointed when people judge the value of a computer scientist’s work more by its economic effects than by its intellectual effects. I envy astronomers: People expect them to study stars because stars are beautiful and interesting?
Alas, people these days rarely measure a computer scientist by standards of beauty and interest; they measure us by dollars or by applications rather than by contributions to knowledge, even though contributions to knowledge are the necessary ingredient to make previously unthinkable applications possible.
RM:
On your website you mention Douglas Preston’s book Blasphemy which I suppose can be described as science meets religion with a side order of politics. Do you think technology or science has a greater chance of discovering God than man?
DK:
I tend to turn that question around, by trying to consider from God’s standpoint what are the best ways to improve people’s spiritual side. “One size doesn’t fit all.” I don’t want to give Preston’s plot away, but I can say that the book’s final sentence is what made me think that his novel is a masterpiece.
RM:
Do you agree with Ray Kurzweil that one day we will develop the ability to reprogram our biology through nanotechnology using nanobots – blood-cell sized devices in our bloodstream that will keep us living for however long we wish to live? Or is that thinking just too outlandish?
DK:
I’ve never been good as a futurist, but I don’t want to duck your question. I have great respect for Kurzweil, having once played on a grand piano that he built for Marvin Minsky, and having experimented with an amazing machine that he designed for optical character recognition, long before anybody else had done anything similar.
I personally think he underestimates the huge gap between what we know now and what needs to be learned before such life-extending devices are feasible; but then again, he knows a lot more than I.
I do have a more fundamental consideration, namely that long life isn’t the point; I don’t think I should want to live longer than I can be helpful to other people. I’m not for euthanasia, but I don’t want to be, say, put on an artificial respirator if I’m brain-dead.
As Sportin’ Life says in Gershwin’s Porgy and Bess: ‘Methusaleh lived nine hundred years. But who calls dat livin’ when no gal will give in, to no one who’s nine hundred years?’ I may have mis-remembered that lyric, but you get my drift.