I once had the dubious pleasure of working alongside a highly paid contractor who kept asking the trainees brilliant questions such as "How do I process every fifth record?" (and it wasn't because he just wanted to know if they knew).

The truth is that people with no aptitude whatsoever for programming *do* manage to fake their way through the system (often they can describe perfectly how something should be done, without actually understanding what they just said).

I am not a programmer (Of course, I have written some, but it is not my profession) but I consider the "off-by-one" error as very serious. Sure it is an easily corrected bug, but it is also trivial to detect, just walk through a string of length 1 and see it you get 0, 1, or 2. Not doing it means that you are not interested in consistency checking.

In my actual profession (teaching math), one of the biggest differences between the best students and the rest was the willingness of the former to actually look at their answers and see if they made sense. Sometimes checking is easy and sometimes hard, but at least in the easy case, it is unforgivable not to check. And I'll bet that a person who doesn't, can't be bothered, doesn't understand the concept, whatever, will never make a good programmer. I suppose it can be taught, but the best do it automatically. The rest generally pay lip service but skip when no one is looking.

I think this is a good question but I would not hire the person who produces the off-by-one error.

I take it the video will be up on CorpNet, and those of us not lucky enough to work at MS can't see it?

Given that today's college courses emphasize managed code languages like Java instead of C or C++, a simple pointer question like strlen may make some sense to guage their understanding of pointers. It would give you a sense if they have a shot at getting something more complex. This question should not take longer than 5-10 minutes and I'd say that would be time well spent in the interview, especially if you are the first person in the loop.

I agree that some people do the off-by-one error. That's a logic error. What I'm dubious about is people who "just can't write code". I guess they mean they "just can't get simple logic correct".

Steve, the talk is only available internally (just went up today actually). Another advantage of that excellent Microsoft corporate network...

Phil, if they haven't ever done pointers in college then this question does take on a different meaning. But in fact if somebody had not done pointers then presumably the interviewer would accept an answer that did it with array indexing instead. I suppose in that case it is a good question for "Did you really understand what I just explained to you about what a pointer is." I remember in CS 217 in school the first time the professor showed us how to do this kind of stuff with pointers and everybody was amazed and wanted to run off and code up some pointer arithmetic (of course nowadays the compiler generates the same code whether you use pointers or array indexing).

- adam

When I was at Amazon, a popular interview question was "given two sorted arrays of integers, merge them into a single sorted array". It was depressing how many candidates could not solve this; they simply had no idea where to start. The worst was a candidate for an SDE 3 (roughly equivalent to L64+ dev) job who was going on and on about all the complicated projects he had worked on, but was reduced to babbling and flailing incoherently by this simple question.

I'm sorry to say that I have a strong suspicion that there are some people who hold some sort of Computer Science degree who would struggle to write strlen.

Many years ago I had the opportunity to observe some university students attempting a project (they had only done a few months worth of programming before this point though). The part I saw involved storing the frequency of results and many students resorted to using an if-then-else ladder to work out where to store things in the array even after being given the hint that it might be more compact to use an intermediate result as an offset into the array. Some of these students were able to use the "shortcut" after detailed explanation of the constituent parts but never really seemed comfortable with what they had done. I don't think they would have enjoyed using pointers even for basic tasks given that didn't seem comfortable using arrays but I am certain they went on to pass their degree.

On a mildly related note, Joel on Software had a good posting on how you can't use pointer questions in interviews any more:
http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html
(there were a couple of interesting comments over here too:
http://scobleizer.com/2005/12/29/joel-says-teaching-java-is-bad-for-cs-students/
(although I find it hard to believe that MS has issues recruiting C/C++ programmers)

Strlen is a good "weeder" question. I prefer using such questions in a test we ask candidates to write before appearing for interview. If somebody can't code up something like strlen then you know you don't need to spend anytime interviewing them. The thing about simple questions is that you must get it right completely. So an off by 1 error in a trivial question is not acceptable.

Earlier I used to be amazed by how many qualified engineers with a CS degree manage to screw up such simple questions. Now nothing surprises me. :-|

My feeling would be that if there were no "just can't code" people to be revealed by strlen(), then you should ask a more complicated question.

Meaning, if there were a lot of people who just couldn't code, and strlen() would make this obvious in a few minutes, then you could ask it to weed out peope quickly.

But, if problems with strlen() only reveal themselves after the candidate has written up a full solution and failed to debug it--in other words after you've devoted 10 minutes to the problem, of the 55 or so minutes you have--then I think the time is better spent on a more difficult problem that might take 25 minutes total (rather than do strlen() in 10 minutes and then a slighly more complicated problem in the next 15 minutes).

Good discussion, if the question comes up again. Thanks.

- adam