November 16, 2006
Programmers Who Can't Write Strlen()A few days ago I gave a talk at Microsoft on "Choosing Technical Interview Questions". The talk was quite popular, there were 180 registered and 508 people on the waitlist. A timely topic it appears. 180 was the official room capacity but there wound up being 241 people in the audience (people scan in with their cardkey so we get an exact count). And the talk went pretty well if I do say so myself. The video should be up soon (go to mylearning and search by the title, if there is a "View" button then the recording is there).
One of the subjects I touched on was what kind of coding question was too easy to bother asking. The C standard library has a function called strlen() which calculates the length of a string (if you've been hiding under a rock and/or programming in one of them "modern" languages, I'll explain that a "string" in C is just an array of characters, terminated by a character with value zero. So the question could be generalized as "You have an array of elements. Count the number of non-null elements at the beginning of the array"). My feeling is that strlen() is too easy a question to ask during an interview.
When I state this, someone will usually say, "But you'd be amazed how many candidates can't do strlen()". To which my standard reply is, sure, maybe they can't, but you might as well ask a harder question, because the strlen()-inept will also fail the harder one, and you'll be setting the bar a bit higher.
But then I got to thinking. Is there anybody who has, say, a Computer Science degree, that can't code up strlen()? I mean it really is an easy problem in any language. You walk the array until you find a null, and keep track of how many elements it takes to find one. C has pointer arithmetic so if you've got some ninja skillz you don't need to keep a separate counter, you can just figure out how far the null is from the beginning of the array. Which leads to the only possible error I can think of someone making, that is having an off-by-one error in their calculation at the end. I wouldn't call that "can't do strlen()" however, I would call that "having a bug in their strlen()".
Someone pointed me to Steve Yegge's column on phone screens (from back when he worked at Amazon) where he advocates asking candidates a trivial programming problem on the phone. Now he doesn't come out and say "You'd be amazed at some candidates who just can't write code", but he implies it by having that as a "weeder" question. Actually while I would consider some of his examples to be trivial (print odd numbers from 1 to 99, find largest number in an array), I would say that some of the others aren't (reverse a string, sum integers in a file). And I have serious issues with some of the other things he writes in that post, which I won't get into. Suffice it to say that he also seems to be (or seemed to be) in the camp that some programmers just can't write code, although he doesn't actually state that he has seen that.
I asked around the Development Engineering Excellence team and basically got consensus that yes, nobody had actually ever seen someone completely fail at strlen(), it was more that they made small mistakes. OK, that's fine. But for now I will assume that the stories of the strlen() disaster are an urban legend.
Posted by AdamBa at November 16, 2006 11:09 PM
TrackBack URL for this entry:
I once had the dubious pleasure of working alongside a highly paid contractor who kept asking the trainees brilliant questions such as "How do I process every fifth record?" (and it wasn't because he just wanted to know if they knew).
The truth is that people with no aptitude whatsoever for programming *do* manage to fake their way through the system (often they can describe perfectly how something should be done, without actually understanding what they just said).
Posted by: Kevin Daly at November 17, 2006 01:36 AM
I am not a programmer (Of course, I have written some, but it is not my profession) but I consider the "off-by-one" error as very serious. Sure it is an easily corrected bug, but it is also trivial to detect, just walk through a string of length 1 and see it you get 0, 1, or 2. Not doing it means that you are not interested in consistency checking.
In my actual profession (teaching math), one of the biggest differences between the best students and the rest was the willingness of the former to actually look at their answers and see if they made sense. Sometimes checking is easy and sometimes hard, but at least in the easy case, it is unforgivable not to check. And I'll bet that a person who doesn't, can't be bothered, doesn't understand the concept, whatever, will never make a good programmer. I suppose it can be taught, but the best do it automatically. The rest generally pay lip service but skip when no one is looking.
I think this is a good question but I would not hire the person who produces the off-by-one error.
Posted by: marble chair at November 17, 2006 08:47 AM
I take it the video will be up on CorpNet, and those of us not lucky enough to work at MS can't see it?
Posted by: Steve at November 17, 2006 11:40 AM
Given that today's college courses emphasize managed code languages like Java instead of C or C++, a simple pointer question like strlen may make some sense to guage their understanding of pointers. It would give you a sense if they have a shot at getting something more complex. This question should not take longer than 5-10 minutes and I'd say that would be time well spent in the interview, especially if you are the first person in the loop.
Posted by: Phil at November 17, 2006 12:21 PM
I agree that some people do the off-by-one error. That's a logic error. What I'm dubious about is people who "just can't write code". I guess they mean they "just can't get simple logic correct".
Steve, the talk is only available internally (just went up today actually). Another advantage of that excellent Microsoft corporate network...
Phil, if they haven't ever done pointers in college then this question does take on a different meaning. But in fact if somebody had not done pointers then presumably the interviewer would accept an answer that did it with array indexing instead. I suppose in that case it is a good question for "Did you really understand what I just explained to you about what a pointer is." I remember in CS 217 in school the first time the professor showed us how to do this kind of stuff with pointers and everybody was amazed and wanted to run off and code up some pointer arithmetic (of course nowadays the compiler generates the same code whether you use pointers or array indexing).
Posted by: Adam Barr at November 17, 2006 02:20 PM
When I was at Amazon, a popular interview question was "given two sorted arrays of integers, merge them into a single sorted array". It was depressing how many candidates could not solve this; they simply had no idea where to start. The worst was a candidate for an SDE 3 (roughly equivalent to L64+ dev) job who was going on and on about all the complicated projects he had worked on, but was reduced to babbling and flailing incoherently by this simple question.
Posted by: Nick at November 17, 2006 05:49 PM
I'm sorry to say that I have a strong suspicion that there are some people who hold some sort of Computer Science degree who would struggle to write strlen.
Many years ago I had the opportunity to observe some university students attempting a project (they had only done a few months worth of programming before this point though). The part I saw involved storing the frequency of results and many students resorted to using an if-then-else ladder to work out where to store things in the array even after being given the hint that it might be more compact to use an intermediate result as an offset into the array. Some of these students were able to use the "shortcut" after detailed explanation of the constituent parts but never really seemed comfortable with what they had done. I don't think they would have enjoyed using pointers even for basic tasks given that didn't seem comfortable using arrays but I am certain they went on to pass their degree.
On a mildly related note, Joel on Software had a good posting on how you can't use pointer questions in interviews any more:
(there were a couple of interesting comments over here too:
(although I find it hard to believe that MS has issues recruiting C/C++ programmers)
Posted by: Anon at November 18, 2006 03:05 AM
Strlen is a good "weeder" question. I prefer using such questions in a test we ask candidates to write before appearing for interview. If somebody can't code up something like strlen then you know you don't need to spend anytime interviewing them. The thing about simple questions is that you must get it right completely. So an off by 1 error in a trivial question is not acceptable.
Earlier I used to be amazed by how many qualified engineers with a CS degree manage to screw up such simple questions. Now nothing surprises me. :-|
Posted by: Gaurav at November 18, 2006 10:00 PM
My feeling would be that if there were no "just can't code" people to be revealed by strlen(), then you should ask a more complicated question.
Meaning, if there were a lot of people who just couldn't code, and strlen() would make this obvious in a few minutes, then you could ask it to weed out peope quickly.
But, if problems with strlen() only reveal themselves after the candidate has written up a full solution and failed to debug it--in other words after you've devoted 10 minutes to the problem, of the 55 or so minutes you have--then I think the time is better spent on a more difficult problem that might take 25 minutes total (rather than do strlen() in 10 minutes and then a slighly more complicated problem in the next 15 minutes).
Good discussion, if the question comes up again. Thanks.
Posted by: Adam Barr at November 20, 2006 05:40 PM