Chet Ramey: Geek of the Week

The BASH shell is the most popular UNIX command-line scriptable shell. It became the inspiration for PowerShell. As with so many standard components of the Open Source movement, there is a hard-working and dedicated individual who quietly supports the tool over many years. Chet Ramey maintains and extends BASH by himself, and we all give thanks to him for his dedication.

2315-chetramey.JPG

Some people working in IT seem to have no “off switch” – in Japan they call it karoshi and in China it is guolaosi. As yet there is no word in English for working yourself to death, though overworking in the  Information Technology industry is not a new phenomenon. It is so common as to be seldom remarked upon.

Unsung volunteers who do such sterling work in the industry could be undone by wanting to do the ordinary things in life as well, such as going on vacation for example. Thankfully, there are those rare men and women who manage to do both. One man who has managed to do this is Chet Ramey – suitably he lives in a place called Novelty in Ohio.

Not only is Chet assistant director of technology infrastructure at the IT services division of Case Western Reserve University and the manager of it Network Engineering and Security Group, he’s the solo maintainer of BASH, an acronym for the ‘Bourne-Again SHell’ a pun on Stephen Bourne, the author of the direct ancestor of the current Unix shell.

A co-author together with Brian J Fox (the original developer of the GNU BASH Shell) of a book on BASH, Chet told Simple Talk of the challenges involved with being the steward of BASH, where the future of the language might lie and his predictions for computer languages in the next decade.

RM:
Chet how did you get involved with BASH? What was your early experience with it consist of? And why BASH?
CR:
Well, the project started back in 1989. Brian Fox had begun writing BASH and readline (which was not, at that time, a separate library) the year before, when he was an employee of the Free Software Foundation (FSF) The story, as I recall it, was that a volunteer had come forward and offered to write a Bourne shell clone.

After some time, he had produced nothing, so Richard Stallman directed Brian to write a shell. Richard said it should take only a couple of months – famous last words!

At that time, I was doing network services and server support for Case Western Reserve University (CWRU) and was looking around for a better shell for the engineers in my department to use. We were running on a set of VAXstations and (believe it or not) some IBM RTs running the original version of AIX.

Remember, back then, there were no other freely-available shells besides tcsh (an enhanced, but completely compatible version of the Berkeley UNIX C shell (csh) and the BSD csh, but that was hard to get separate from BSD). There were no free Bourne shell clones. Arnold (now Aharon) Robbins had done some work, based on work by Doug Gwyn and Ron Natalie at the U.S. Army Ballistics Research Laboratory, to add things like editing and job control to various versions of the Bourne shell, but those changes obviously required a licensed copy to patch. We had the patch but limited where we could put it. Ken Almquist was in the midst of writing ash, but that had not seen the light of day either.

Charles Forsyth’s PD clone of the V7 shell, which became the basis of the PD ksh, was available, but little-known, and certainly didn’t have the things I was looking for: a command-line editing with emacs key bindings and job control.

I picked up a copy of BASH from somewhere, I still can’t remember where, and started to work on it. Paul Placeway, then at Ohio State where he was the tcsh maintainer for many years, had gotten a copy and was helping Brian with the readline redisplay code, and I got a copy with his changes included. (If you look at tcsh, the BSD `editline’ library, and readline, you can tell they all share a common display engine ancestor.) I took that code, made job control work, fixed a bunch of bugs, and, after taking a deep breath sent the changes off to Brian. He seemed impressed, incorporated the fixes into his copy, and we began corresponding.

When Brian first released BASH to the world, I fixed all the bugs people reported and fed the fixes back to Brian. We started working together as more or less co-maintainers, since Brian was looking for someone to take over some of the work. I wrote a bunch of code and the first cut of the manual page.

After a while, Brian moved onto other things, and BASH development stalled. I still needed to support BASH for my department, and I liked working on it, so I produced several `CWRU’ BASH releases. Brian came back into the picture for a while, so we folded our versions together, resulting in BASH-1.10. By the time BASH-1.14 was released, Brian had eased out of the development, and I took over.

The name itself is an acronym, a pun and descriptive. As an acronym, it stands for Bourne-again shell, referring to its objective as a free replacement for the Bourne shell. As a pun, it expressed that objective in a phrase that sounds similar to born again, a term for spiritual rebirth. The name is also descriptive of what it did, BASHing together the features of sh, csh and ksh.

RM:
Can you remind me how BASH came about?
CR:
Well it was one of the key pieces of the GNU system. The FSF had been producing well-respected tools like emacs and early versions of GCC and GMAKE, but you could not hope to ever achieve their goal of a complete Unix replacement without a Bourne-compatible shell. To satisfy the FSF’s other requirements, the shell would have to have acceptable licensing, which dictated its from-scratch implementation.

Since the Posix folks had started their Unix standardization work (Stallman invented the word “Posix”), it made sense to make whatever shell the GNU project constructed be compatible with the standard. The way I heard it, Stallman underestimated the work involved considerably.

RM:
Was there a particular problem that BASH aimed to solve?
CR:
The primary problem was compatibility. BASH, at least at the beginning, wasn’t looking to break new ground. In the years since, of course, features originating in BASH have found their way into other shells and into Posix, but it didn’t really start out that way.

The Bourne shell was intended to solve a number of problems and limitations with the Thompson/Mashey shell, which was the standard shell through the Sixth Edition of Unix. Steve Bourne did extensive reworking of the Sixth Edition shell’s syntax with the goal of improving its programming capabilities (flow control, quoting, variables and so on).

RM:
You took over as the language’s primary maintainer some years ago, do you have a team around you or is it very much a solo effort?
CR:
It’s a solo effort. I get great bug reports and occasional code contributions from around the net, but virtually all of the development and bug fixing I do myself.
RM:
Is compatibility the toughest job in maintaining the language?
CR:
It’s really balancing compatibility with historical versions of sh and the Posix standard. Folks have different ideas of what “the Bourne shell” means; usually it means “whatever sh does on my system,” since most people don’t have the resources to support access to lots of different Unix systems. The problem with that is that the “Bourne shell” changed over time: the 7th Edition shell differed from the SVR2 version (a major rework), which did things differently than the SVR3 sh, which was different from the SVR4 and SVR4.2 versions, and so on. Individual vendors also mixed in their own changes.

To further complicate matters, the Posix committee decided to depart from historical practice in a number of different areas, sometimes unintentionally. Since BASH is intended to be Posix-conformant, I will choose Posix compatibility when there’s a dispute, with a few exceptions.

The other sometimes-tricky piece is BASH’s backwards compatibility with itself. The longer BASH lives, and the more people who use it, the harder it gets to fix things I consider mistakes. Not everything that goes into BASH is perfect the first time.

RM:
Did you write the BASH reference manual to support local users or was it something you thought should be done anyway? What’s the date of the next release?
CR:
Certainly both. I needed a manual to support my local users, so it would have been done anyway, but I wrote it with an eye to wider distribution.

One problem was that the manual I distributed described features existing only in my local version. That was eventually resolved when the development threads were unified.

I’m planning on releasing BASH-4.1 sometime in late 2009 or early 2010. The time between releases, even minor “point” releases, is around a year. It takes that much time for vendors to replace an essential part of their core system.

RM:
Is there a strong relationship between the original Bourne Shell and the Bourne-Again Shell?
CR:
In terms of the language and its syntax, yes. All variants of the Bourne shell share a great deal. There is a linear relationship: the original Bourne Shell was very influential, the various System V shell releases preserved that heritage, and the Posix committee used those versions as the basis for the standard they developed.

Certainly the basic language syntax and built-in commands are direct descendants of the Bourne Shell’s. BASH’s additional features and functionality build on what the Bourne shell provided. As for source code and internal implementation, there’s no relationship at all.

RM:
Where do you think BASH’s future will be?
CR:
BASH will continue to evolve both as an interactive environment and a programming language. I’d like to add more features which allow interested users to extend the shell in novel ways. The programmable completion system is an example of that kind of extension. BASH’s evolution has always been user-driven, so it will ultimately be up to the feature requests that come in.
RM:
Do you think BASH has brought a lasting legacy to the web?
CR:
I think it’s brought a lasting legacy to open source, if not necessarily the web. It’s widely-distributed and widely-used. Linus Torvalds has said that BASH and gcc were the first significant programs he ran on his newly-implemented Linux kernel back in 1991, for what that’s worth!
RM:
Will BASH continue to evolve as both an interactive environment and a programming language?
CR:
Yes. It serves both purposes. BASH has always evolved in both areas: BASH-4.0 (and previous releases), for instance, brought significant improvements in both, and BASH-4.1 will continue that. What do you think of Microsoft’s efforts to take part in the open source community? Do you think they are sincere in their efforts or do you see it as some sort of embrace-extend-extinguish approach?

I think the open-source efforts of the individual Microsoft participants are genuine. Microsoft management, on the other hand, may have different priorities.

RM:
You’ve had some dealings with Richard Stallman in the past, is open source a religious issue for you or was it chosen on a whim?
CR:
I wouldn’t exactly call it a whim. It’s not a religious issue for me; I just want to see the code I write used as widely as possible. For BASH and readline, I didn’t choose the licensing, the software already existed and I worked with what was. I’m comfortable working within the parameters of the FSF’s free software philosophy.
RM:
Where do you see computer programming languages heading in the future, particularly in the next five to 10 years?
CR:
I see increased dynamism, allowing programmers to do more and more complex things on the fly, especially over the Web. The advances in hardware allow interpreted code to run faster today than compiled code on some systems available when I started work on Bash.