Pipe: How the System Call That Ties Unix Together Came About
It’s an everyday command in the life of developers, sysadmins, and Unix lovers everywhere. So it’s remarkable to remember that Unix’s pipe command was implemented in a single day, representing not only a great moment in computing history, but also a uniquely important moment for its profound impact on the culture of Unix.
And it changed the way we program ever since.
Envisioned in the 1960s
In fact, it was more than 50 years ago that Doug McIlroy, who headed Bell Labs’ famous “Computing Techniques Research Department” from 1965 through 1986, wrote, “We should have some ways of coupling programs like garden hose.” This was 1964 — he’d pecked the words out on a typewriter — saying this method would let programmers “screw in another segment when it becomes necessary to massage data in another way.”
Michael S. Mahoney, a science historian at Princeton University who created an oral history of Unix in 1989, argued that the day pipes were actually created marked a significant turning point for the Unix philosophy. So when he finally tracked down McIlroy, the historian said there was a specific reason why he’d wanted to ask McIlroy about pipes: because Unix co-creator Dennis Ritchie had said that “not only was it your suggestion, but indeed, he suggests, at your insistence.”
McIlroy (left) with Dennis Ritchie in 2011
McIlroy acknowledged that pipes “was one of the only places where I very nearly exerted managerial control over Unix, was pushing for those things.” McIlroy remembered that yes, he’d kept bringing it up internally at various points over the next nine years, until finally one day in 1973, Ken Thompson had said “I’m going to do it.”
“He was tired of hearing all this stuff… He didn’t do exactly what I had proposed for the pipe system call; he invented a slightly better one….” Since most of the common commands still took their input from files — like grep and cat — “he went in and changed all those programs in the same night. I don’t know how … And the next morning we had this orgy of one-liners…
“[T]hat was absolutely a fabulous day the next day.”
And in their ensuing conversation, the historian establishes that this was the moment that truly gave birth to the idea that Unix is a collection of tools.
Mahoney: Was the notion of toolbox there before pipes …?
McIlroy: No.
Mahoney: Or did pipes create it?
McIlroy: Pipes created it.
Mahoney: Unix looked different after pipes?
McIlroy: Yes, the philosophy that everybody started putting forth, “This is the Unix philosophy. Write programs that do one thing and do it well. Write programs to work together. Write programs that handle text streams, because that is a universal interface.” All of those ideas, which add up to the tool approach, might have been there in some unformed way prior to pipes, but they really they came in afterwards.
Five years after that fateful day, McIlroy even carefully spelled out “the Unix philosophy” in the “Bell System Technical Journal,” a philosophy which argues for individual programs that do one thing well. “Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information.”
Amazingly, a copy of the 41-year-old document is still available online.
Though Mahoney’s interview suggests that the original symbol for pipes was “>,” in the 1994 book, “A Quarter Century of Unix,” McIlroy remembers when Thompson decided the symbol for pipes should instead be “|.” Thompson made the change “for a talk in London, because he couldn’t bear to reveal my ugly syntax.”
Kernighan’s Memories
It’s hard to overstate the impact that this new functionality brought to the world of Unix. Mahoney also later tracked down Brian Kernighan, another of the operating system’s co-creator, who also strongly underscored the importance of that one moment in time. He recalled:
All of a sudden there was the vertical bar and just everything clicked at that point. And that was the time then I could start to make up these really neat examples that would show things like doing — you know, running who and collecting the output on a file and then word-counting the file to say how many users there were and then saying, “Look how much easier it is with the who into the wordcount,” and then running who into grep, and starting to show combinations that were things that were never thought of and yet that were so easy you could just compose them at the keyboard and get ’em right every time.
And that’s I think when we started to think consciously about tools, because then you could compose the things together, if you had made them so that they actually worked together. And that’s when people went back and consciously put into programs the idea that they read from a list of files, but if there were no files they read from the standard input so that they could be used in pipelines.
It wasn’t just another tool. It quickly became indispensable. In 1985 the team’s co-worker Dick Haight also shared his memories of that specific day for an article in The Unix Review by August Mohr (which was cited in the 1996 ebook “Netizens: On the History and Impact of Usenet and the Internet.”)
“It was clear to everyone, practically minutes after the system came up with pipes working, that it was a wonderful thing. Nobody would ever go back and give that up if they could.”
It looks like for those who were there, it was a moment in time that was destined to be fondly remembered forevermore.
_and_Dennis_Ritchie_at_PDP-11_(2876612463).jpg){kind=link}