"We could even watch people do hands-on work before hiring them"

I think a lot of groups already do this. They just call it contracting.

It seems especially odd for internal candidates. We can't just look at their check-ins and talk to their peers?

One thing I liked at one company was solving a sample problem together: "let's sketch out a payroll system/online shopping cart/mail form/whatever". It was important to pick something you yourself hadn't done, so you wouldn't know "the answer", and that you would actually contribute to the sample solution.

The hiring process for many jobs is pretty ad hoc but in programming it's probably particularly awful.

You would think a company like MS, Google or Nvidia would carefully study their own hiring process and check how well people do from different ways of hiring. Of course creating a metric for measuring the success of new hires would be difficult.

Certification would help, but don't forget it would cost a lot. There would be some basic certification and then a swathe of specialist stuff. You would probably need quite a bit of certification. Java's levels would just be a start. But it might be worthwhile. But then again, how many people have you met who think that Java's certification works well? What would?

Phil, is that true? I know we do hire former contractors, but I wonder how much their previous work is taken into account. Obviously if it is for the same group it has an effect, but do hiring managers contact the former manager of the mentor if they were in a different group?

I actually wrote a thinkweek paper about studying how different kinds of interviewing worked out...didn't get much traction however. I can send the link to anybody who emails me at work.

- adam

You should read, if you haven't already, "How would you move Mount Fuji". The first half of the book explains the history that has led us to the sorry state of technical interviewing today. A lot of it is due to legal considerations - for example it's very hard to get a US company to sey anything, good or bad, about a previous employee.

One of the most interesting parts of the UX discipline is the baked-in 'portfolio review' concept for every interview. The candidate presents the meaningful pieces of their work from over a time period of their choice - to show breadth, depth, creativity, presentation and preparation and speaking skills, etc. It is *such* a great way to get a better picture of a candidate than just a set of one-hour interviews.

Regarding contractors, speaking for myself any time i've hired anyone, if I have a contact of someone @ msft they worked for in the last two years, i'll ping that person for a reference.

Interesting, Adam. I came to Microsoft shortly after I quit my medical residency; I was too much a hacker at heart and was not enjoying medicine (I've since left MSFT).

I hope that, if we DO implement a broad certification for SDE's, it will not become the tyranny of a guild as it has in medicine. Medicine struck a Faustian bargain with government--"If you protect us from competition, we'll police ourselves internally with certifications and submit to the FDA". It's created a far-from-free-market money-pit.

Programming is wonderful precisely because it is open to all, but for the most part only the truly talented survive and ascend. The cost of becoming a "programmer" is almost zero; and one's rise is limited only by intellect and drive.

Contrast this with medicine, where the certification process is highly standardized but extremely costly. Most emerge with well over $200K in debt and about 8 years behind their engineering peers. As you noted, interviews consist not of skill-evaluation but rather personality testing. Patients too have been deceived into believing that medical training is a commodity, and do not have the information or tools to reward and punish according to skill; for most of us, we judge our doctors by personality and whether they're "in our insurance network".

This leads to sub-optimal outcomes. Any honest doctor will tell you there is a wide variation in standard of care by geography, hospital, medical school, practice type, and so on; yet patients are not privy to this.

Programming is not a meritocratic paradise, but I do think that our lack of certification actually forces us to evaluate ourselves more on "outcome" than in medicine.

Cheers!
Michael Stuart