« The New New Microsoft Store | Main | Atul Gawande Visits Microsoft »

January 05, 2010

The Importance of Experimenting

Atul Gawande recently wrote a New Yorker article entitled "Testing, Testing". He explains how agriculture in the United States became dramatically more efficient in the early part of last century. The government did not come in with a central plan for how to run farms; instead, it found farmers willing to experiment with new ideas, and then it used their successes to promote the ideas that worked. As a quote in the article explains, "What a man hears he may doubt, what he sees he may possibly doubt, but what he does himself he cannot doubt." Gawande is using this example to explain that the proposed US health care bill, which is small on big ideas for reducing costs, but big on small pilots for reducing costs, is not such a bad thing. But as is often the case with the medical profession (see here and here and here and here), there are a lot of parallels with software development.

In Engineering Excellence at Microsoft, I think we often come across as being the central government telling other people what to do, with no experimental results to back it up. Here is a recent comment on Mini-Microsoft: "What's up with the EE team? I was listening to a presentation by Alan & most of his ideas are old (some blatantly taken from James Whittaker). Can we get some originality & accountability in that team?" Now, we have been talking about piloting more things ourselves (and encouraging others to pilot more things), but the point does hit home, and is especially important at Microsoft, about which the quote (from Gawande's article) "there was a deep-seated fear of risk and the uncertainties of change; many farmers dismissed new ideas as 'book farming.'" could be directly applied (c/farm/engineering team/). People at Microsoft have a virtually infinite ability to convince themselves that their team faces unique challenges, and therefore they have nothing to learn from other teams that have been successful (I think Steven Sinofsky's book is partly about the challenge of fighting that belief among the different teams in Windows).

Gawande writes, "There are, in human affairs, two kinds of problems: those which are amenable to a technical solution and those which are not." As it happens I recently took a course on "Adaptive Leadership", based on the book Leadership on the Line, which makes the same distinction--it talks about "technical challenges" and "adaptive challenges", with the adaptive ones being the tough ones that don't have a known solution. My favorite quote from the book is: "Leadership is the art of disappointing people at a rate they can tolerate" (see examples under "health care bill, disappointment with Barack Obama over"). The claim is that solving adaptive challenges, by their nature, is going to be disruptive: things that can be solved by business-as-usual are by definition merely technical challenges. I think many in EE expect that we can figure out all the answers on how to develop software and all we have left is the technical challenge of spreading the word. But the more time I spend here, the more I am convinced that the pilot program approach that Gawande talks about is a much better way to go. I think this will disappoint some people; hopefully they will be able to tolerate it.

Posted by AdamBa at January 5, 2010 02:15 PM


Insightful. Whatever MS is doing obviously isn't working very well and hasn't for most of the last decade. So it's hard to understand the continued resistance to a change of approach. Hasn't it dawned on the average employee yet that in virtually every market where MS competes it has either lost (actually or effectively), or is steadily losing share? Weren't last year's results and the layoffs a sufficient wake up call for even the most die-hard of remaining believers?

The alternative approach you're suggesting is also similar to that being pursued by Gates with his own foundation (i.e. find small teams who are already getting results and then provide additional funds to scale it out).

Posted by: Bob at January 6, 2010 09:58 AM