(I originally posted this on my MSDN blog.)
I’ve been participating in more conversations internally about promoting a team-oriented culture at Microsoft. Microsoft has a strong individual-oriented culture which works well for many things but doesn’t work so well for agile software development. The question of paying (at least partly) based on team results rather than individual results came up, and it’s an interesting thing to think about.
Many groups at Microsoft strongly focus individual-contributor evaluations on individual results, not team or product results. For instance, a developer might be evaluated on the number of product features he delivered and the number of bugs found in his code. That’s understandable in the sense that that individual results are easy to define and easy to measure, but a huge problem in that there’s no guarantee that successful execution on a random collection of individual commitments will result in working software.
That is to say, all of the PMs can successfully write specs, all of the devs can successfully write code, and all of the testers can successfully find bugs, and yet the business fails to make money because the parts didn’t synthesize together into a cohesive whole. Of course when you get to the senior manager level you start to see commitments that explicitly say, “Ship a successful product,” but ICs don’t usually have that in their goals and aren’t incentivized to make decisions that might promote the “ship a successful product” goal at the expense of their own individual goals.
The root issue here is the general failure mode of all metrics. If you define a metric and evaluate people on it, then people will optimize for delivering on that metric. When your metrics represent only secondary attributes rather than your true core goal, then you’re at high risk of people optimizing in ways that actually hurt your core goal rather than help it. Metrics are useful in direct proportion to how closely they represent the core goal. At Microsoft, our core goal is to deliver successful software products. Our evaluation and compensation system ought to be tied as closely to that core goal as we can possibly get it.
Unfortunately our core goal is rather fuzzy. Well, I guess measuring achievement of the core goal is pretty straightforward but mapping the ultimate end result back to the individual contributions that people made to get you there is really, really hard to do in an objective, measurable fashion. When we try to solve that problem we inevitably end up measuring indirect effects rather than primary causes, which makes people optimize on the wrong things, and we’re right back where we started.
It would be great if we could come up with an evaluation and compensation system that would explicitly encourage people to stop and ask themselves, “What is the one thing I could do today that would have the greatest impact on shipping a successful product,” rather than asking themselves, “What did I write down in my commitments last summer that I haven’t done yet?” That difference in mentality may be subtle but it has far-reaching implications in behaviors and the choices we make.
SaintGimp 