« July 2008 | Main | September 2008 »

August 29, 2008

Eureka Moments and Interviews

I've been meaning to blog about this (or anything) for a while. In the July 28 New Yorker, there was an article titled "The Eureka Hunt", about where sudden insights come from. Some key quotes:

• "There is something inherently mysterious about moments of insight...Such tales all share a few essential features, which psychologists and neuroscientists use to define 'the insight experience.' The first of these is impasse: before there can be a breakthrough, there has to be a mental block...another key feature of insight [is] the feeling of certainty that accompanies an idea."
• "Schooler had demonstrated that it was possible to interfere with insight by making people explain their thought process while trying to solve a puzzle--a phenomenon he called 'verbal overshadowing.' This made sense to Jung-Beeman, since the act of verbal explanation would naturally shift activity over to the left hemisphere, causing people to ignore the more subtle associations coming from the right side of the brain."
• "Jung-Beeman began searching in the right hemisphere for the source of insight in the brain. He decided to compare puzzles solved in moments of insight with those solved by methodical testing of potential solutions, in which people can accurately trace their thought process and had no sense of surprise when the answer came. Unfortunately, all the classic puzzles developed by scientists to study insight required insight; if subjects didn't solve them in a sudden 'Aha!' moment, they didn't solve them at all."
• "Jung-Beeman needed to develop a set of puzzles that could be solved either by insight or by analysis....he eventually settled on a series of verbal puzzles...In a C.R.A. word puzzle, a subject is given three words, such as 'pine,' 'crab,' and 'sauce,' and asked to think of a word that can be combined with all three--in this case, 'apple.'"
• "The resulting studies...found that people who solved puzzles with insight activated a specific subset of cortical areas...The first activities activated during the problem-solving process were those involved with executive control...What happens next is the 'search phase' as the brain starts looking for answers in all the relevant places...The search can quickly get frustrating...'Almost all of the possibilities that your brain comes up with are going to be wrong,' Jung-Beeman says, 'And it's up to the executive-control areas to keep on searching, or, if necessary, change strategies and start searching something else.'"
• "The insight process, as sketched by Jung-Beeman and Kounios, is a delicate mental balancing act. At first, the brain lavishes the scarce resource of attention on a single problem. But once the brain is sufficiently focused, the cortex needs to relax in order to seek out the more remote associated in the right hemisphere, which will provide the insight. 'The relaxation phase is crucial,' Jung-Beeman said. 'That's why so many insights happen during warm showers.' Another ideal moments for insights...is the early morning, right after we wake up. The drowsy brain is unwound and disorganized, open to all sorts of unconventional ideas."
• "One of the surprising lessons of this research is that trying to force an insight can actually prevent the insight. While it's commonly assumes that the best way to solve a difficult problem is to focus, minimize distractions, and pay attention only to the relevant details, this clenched state of mind many inhibit the sort of creative connections that lead to sudden breakthroughs."
• "'There's a good reason Google [figures he would use THEM as the example - ed.] puts Ping-Pong tables in their headquarters,' Kounios said, 'If you want to encourage insights, then you've got to also encourage people to relax.'"

So, how does this relate to one of my favorite topics for rumination, namely how Microsoft does its interviews?

I've complained in the past about brainteaser interviews and how unfair they are, and the article seems to support that: a stressful situation like a job interview seems about the worst time to ask somebody to come up with an "Aha!" moment, being a prime example of the "clenched state of mind." I think a lot of people (the ones who got past the "Well, *I* know the answer" self-justification) rationalized asking such questions by figuring that people who couldn't get them just needed to focus harder, which the article implies is incorrect.

BUT I have been struck by two trends, which may be related. Microsoft has (largely) stopped asking brainteaser questions since around the year 2000; and Microsoft's stock hasn't gone up much since around the year 2000. Is it possible that these two are related--that brainteaser questions are actually a better way to interview developers than whatever replaced them?

I'll point out that some people did get "Aha!" brainteaser questions right, even during the stress of an interview. I was one of those people; for whatever reason, I was able to relax my brain enough for the right neurons to fire in the proper hemisphere. The question then becomes, is this the same kind of brain activity that you need to be a good developer? You could argue either side of this; certainly debugging a stress failure two days before you are supposed to ship is a stressful situation, the solution is often an "Aha!" realization of how the bug occured, and there may not be time to go take a warm shower while you are trying to figure it out. But at the same time, a lot of the skill in programming is in doing good design upfront, and given the current state of software design, this is also a series of "Aha!" moments (The Soul of a New Machine has a chapter titled "Wallach's Golden Moment" which describes such an event during the design of the machine; the situation involves somebody thinking about a problem all day, then coming back the next morning and "Suddenly, without thinking about it" having a moment of insight; at the end, as Kidder writes, "As for Wallach, after he had drawn the diagram, he started at it, wondering for a moment, 'Where did that come from?'"). Perhaps we want people who can come up with "Aha!" moments, but it's OK to give them time to shower (probably not a bad thing to encourage for general programmer hygiene). Thus, a good approach to brainteasers might be to pose one early in the morning, and then, after the candidate has spent some time on recruiting shuttles and waiting in building lobbies, come back at the end of the day and see if they have thought of anything.

In his book How Would You Move Mount Fuji?, William Poundstone writes about the approach to solving tricky problems: "To deal effectively with puzzles (and with the bigger problems for which they may be a model), you must operate on two or more levels simultaneously. One thread of consciousness tackles the problem while another, higher-level thread monitors the progress. You need to keep asking yourself 'Is this approach working? How much time have I spent on this approach, and how likely is it to produce an answer soon? Is there something else I should be trying?'" There is a nice parallel with the fairly conscious process that Poundstone describes and the subconscious one that the brain researchers describe in seeking eurekas. I pulled that quote from a review I wrote of the book, which summarizes my thinking on interviews as of mid-2003. I generally still agree with that (except for some waffling on the brainteaser question), and in the context of the discussion here, I like my point that brainteasers might be useful so that you can observe how the candidate approaches a difficult and ill-specified problem with an unbounded solution space (which, of course, is NOT how people used them; they wanted the right answer).

Here's a more radical thought. In the Slashdot article I quote James Fixx, in More Games for the Superintelligent: "While the less intelligent person, unsure of ever being able to solve a problem at all, is easily discouraged, the intelligent person is fairly sure of succeeding and therefore presses on, discouragements be damned." Perhaps what is really happening is that confident people assume they will succeed, which relaxes them (and their cortex) enough during the interview to actually allow them to succeed. Thus confidence becomes a self-fulfilling prophecy--not only during the job interview, but during the job itself. It would seem anathema to modern interviewing techniques to worry not about the answer the candidate gets OR about how they approach the problem, but simply whether they THINK they will be able to solve it--effectively boiling the interview down to the single question "Do you think you are an awesome developer." Maybe asking brainteasers is effectively the same thing, because confident people will solve the brainteasers, and later they will find the tricky bugs and come up with the tricky design breakthroughs. Maybe the awesomeness question IS the only one you need to ask during an interview. And since you CAN judge confidence in the first 15 seconds, maybe those quick judgements are correct--it IS all about the firm handshake and the level stare.

I'm also intrigued by the notion of questions that could be solved either by brute force or by insight. We could ask candidates such questions, figuring that most people would at least get an answer and therefore not hate us for asking impossible questions; then we could ask the candidate HOW they solved it (or just hook them up to a brain scanner first), and based on the answer to THAT seemingly innocuous question, we could do something clever (precisely what, I haven't figured out yet).

By coincidence (I think), that issue of The New Yorker also had an all-time classic article called "Dr. Kush", about California's medical marijuana laws and the effect on the pot industry in the state. The author's approach was to travel the length of the state, find people involved in various aspects of the business, and then spend some time getting baX0Red with them. From the haze of the article emerges the fact that there are two strains of the cheeb, indica and sativa: "Indica is a heavier, numbing drug; sativa is better for doing creative work or listening to music." OK, I can't be the only programmer who reads the magazine; there must be somebody at Google right now who is also putting two and two together. Surely the Big G can find some doctor who would prescribe headies as a suitable cure-all for the niggling physical ailments that befall programmers who spend too much time chained to a desk. A blanket prescription for the whole Googleplex would seem in order: dump those Ping-Pong tables, bring in some papasan chairs, and let a thousand cortexes relax. And the best part is, they already have all the free food they could want.

Posted by AdamBa at 07:36 PM | Comments (9)