« Driving on the Other Side | Main | "Hard Code" Kerfuffle »

April 29, 2008

The Myth of the Lone Programmer

There was a recent article on Slashdot with the salacious title Donald Knuth Rips On Unit Tests and More, implying that the Lord of Algorithms thought unit tests were a waste of time. If you read the actual interview, you will see that Knuth is simply saying that in his own experience writing code, he has never found much use for unit tests.

My interpretation of this is that Knuth has never had to write a particularly large piece of software where he was not the only author. Because that is where unit tests can really help: preventing somebody from accidentally breaking somebody else's code when they make a change. If you are the only author, you may have a good enough idea of the code in your head to prevent such errors (although it would be somewhat conceited to think you would never do it accidentally).

Knuth's idea on how people should write code is an idea he came up with called literate programming, which involves creating a document that describes the code and also can have the source code extracted from it. Knuth introduced literate programming in a language called "WEB", which he used to write TeX (my father says that Knuth chose the term "web" because back in 1981 it was a three-letter word that had no existing computer-related meaning). Literate programming is not a bad idea, but the notion that it would solve modern programming problems is simplistic--unless you are only thinking about software written by one person (as an aside, there was somebody, I don't know who, who used to work in Engineering Excellence and misinterpreted "literate programming" to mean embedding comments in your source code that could be extracted by a tool to produce documentation--what programs like Doxygen and Sandcastle do. So in our slides there are various incorrect uses of the term, every time we talk about documentation generators).

In my class this week I happened to have somebody who worked with me at Softimage back in 1996, and we were reminiscing about some of the programmers who worked there. In particular, a couple of the key architects on the product were "lone wolf" type programmers who spent their time cranking out code and didn't interact with people much, except possibly to point out things they were doing wrong. This led to various zany designs and rules that didn't make much sense, but which it was a losing proposition to try to argue against (the most notorious of these was the stipulation that include files in C++ sources had to be alphabetized by filename; it's a strange enough rule that it's hard to think of a real-world analogy, but you might consider a university where you were required to take all your courses in alphabetical order by course name).

You still hear people gooing and gahing over programmers like that: "they don't talk to anybody, but they sure write a lot of awesome code!" In prior eras at Microsoft such programmers were celebrated, and then we went through a period where they were tolerated, but now I think the pendulum has swung, correctly, to view such people as negatives on a team.

Looking back we probably should have cashiered those guys out of Softimage, but from the inside it always seems like it's worth sticking with them in the short term because the temporary pain of losing them would be so great...but it's not worth it. Hatching a giant code egg all by yourself, even if it is "good" code (and it very likely isn't), just isn't a key part of being a software architect. The job is really all about helping others succeed, growing the skills of the team, representing your team in interactions with other teams, and so on. Sitting in your office with the door closed accomplished none of that. Arguably it would be a case of "what got you here won't get you there", except that in retrospect it shouldn't even have gotten them here.

Posted by AdamBa at April 29, 2008 09:46 PM

Trackback Pings

TrackBack URL for this entry:
http://proudlyserving.com/cgi-bin/mt-tb.cgi/661

Comments

Knuth claimed that "web" was the ONLY three letter English word that had never been used in a computer context. He assigned a grad student to write the very first version of TeX (TeX 1, of course). He did it over around 1981 and that is where he created web. A web program reads like a novel (sort of) and does mix the code and comments. A program called weave then produces both the documentation and a Pascal (later C) program. Looking back 27 years we can say that web was a failure (but TeX certainly isn't).

I once wrote a kind of TeX interpreter. It didn't do things like justifying lines (I am still not convinced that is worth the effort) or hyphenating words, but its main flaw is that it was married to one particular printer (an Epson, I think it was an FX-80) and its capability to produce half-height letters, slanted letters (using delays on the pins), bold letters (double strike) and download a whole other font of 256 characters and switch readily between them.

I have never collaborated with anyone on a program, so I keep clear of the issues Adam has raised.

Posted by: marble chair at April 30, 2008 05:01 PM

I fully agree with the problems these egg hatcher create. It is very important that the rest of the team can see the growing of a project.

However the same problem exists on a bigger level, where you have teams instead of programmers. Each one works on something, that is merged when it is ready tested and profiled. This creates same kinds of problems but on a much larger scale.

Posted by: Ivan at May 1, 2008 05:48 AM

Here is an interesting interview with Knuth that readers of this blog might find interesting:
http://www.informit.com/articles/article.aspx?p=1193856&rll=1
It is rather long and I haven't finished reading it.

Posted by: marble chair at May 2, 2008 05:26 AM

Sunlight is the best disinfectant.

I run into "Lone Programmers" all the time and I agree, while brilliant their opaque coding style adds a degree of risk to the project.

In fact I might suggest that unless you have 2 other programmers on staff that understand the code of the "Lone Programmer" you have just created a liability difficult to extracate yourself from.

Sometimes the LP does this on purpose and burrows themselves deep into the company to the point where they seldom have to do much of anything. Almost as if they are paid on retainer in case their code breaks.

Posted by: Aaron Zalewski at May 2, 2008 06:16 AM

I work on a team of around 30 developers and 10 QA engineers. We all informally review each others submissions as they are put into the source tree (in addition to QA running a comprehensive auto-test suite in the code and on the GUI on each official daily build).

10-15 minutes at the beginning of the day of looking over everyone else's code from yesterday for stupid mistakes leads to great stability and easily catching large problems in the long run. It also gives me something to do while I drink my coffee.

The Lone Wolves have nowhere to hide in such a system.

Posted by: c at May 3, 2008 02:06 PM

In about 5 years working in the software industry, I have yet to find a programmer who locks himself away and produces good code. I have seen a LOT of incredibly bad code that may have been praised at certain times, but I can't say that I have ever seen their code last any test of time. I'm not sure how anyone can think they're brilliant if they never communicate or write code that others can understand. The truly brilliant programmer could lock himself in a room for a week and produce code that anyone could understand and maintain. Even then, brilliant programmers wouldn't want to do that.

Posted by: Brian at May 3, 2008 07:53 PM

FYI, there is a good reason to alphabetize your includes in C++. Sometimes bugs will appear or disappear depending on the order of includes. If some files include .h files in different orders, you can see these bugs appear and disappear arbitrarily far away in the source code depending on which of those files get included first. By alphabetizing your includes you avoid this seeming nondeterminism.

This may only show up in rather large codebases, but it isn't as pointless as it seems.

Posted by: Kevin at May 5, 2008 12:44 AM

Don't worry about Lone Programmers -- they're actually quite rare. Much more common is mediocre or poor programmers. The damage these folks can do is fantastic. Code reviews don't help. Well, they would help but when code painfully thrown together by a mediocre programmer for a month is reviewed and management is told that it's so bad that it needs to be rewritten, management will ignore the recommendation and have the code checked in anyway. So after a while code reviews are halted because there really is no point.

Posted by: Neil Weber at May 5, 2008 01:05 PM