Month: January 2010

(I originally posted this on my MSDN blog.)

Someone asked a question on an internal distribution list to the effect of:

“My team is thinking about making unit testing a standard practice.  The technical benefits are obvious, but management is asking questions about the impact to the business.  How badly will productivity suffer in the short term?  What’s the ROI of unit testing?  Are there metrics or published studies I can use?”

My answer:

You may be able to find some hard statistics in various reports, but practical problem is that the variance from team to team can be huge so there’s no way to take the results from some other team and use it to predict with any accuracy what will happen to your team.  There are a couple of things that can be said with certainty.

First, introducing unit testing will decrease your team’s productivity in the short term, no question.  By how much?  That depends on the team and how much they resist learning new ways of doing things, and it also depends on management and how much they really mean it when they say they want developers to write unit tests and they’re willing to pay the upfront cost.  The transition could go relatively smoothly or it could be really ugly depending on how cooperative people are feeling.

It also depends heavily on the existing architecture of your code.  Effective unit testing requires your code to be organized according to basic principles of good architecture; i.e. high cohesion, loose coupling, and everything that goes along with that.  There’s nothing like unit testing to point out that your supposedly good architecture is actually terrible.  Introducing unit testing may ultimately mean a lot of refactoring of your existing code.  It doesn’t have to happen all at once but it may have to happen at some point.

Second, the payoff won’t be felt in the short term.  The payoff is long-term after you get your architecture straightened out, after you have a good base of unit tests, and after everyone gets fully on board with the new way of doing things.  Then you’ll see fewer bugs being written in new code, fewer regression bugs in old code, and less incremental cost for adding or changing features.  Your code base won’t “rot” as fast.

Unfortunately the value here is difficult to quantify in hard numbers.  The amount of value you earn back depends largely on whether the team is honestly committed to unit testing or if they’re just paying lip service to it, and it depends on whether you’re doing unit testing correctly or not.  It’s certainly possible to write unit tests badly just like it’s possible to write code badly.  If at all possible you should invest in bringing in an experienced unit testing expert who can show you how to do it correctly and what pitfalls to avoid.

 

(I originally posted this on my MSDN blog.)

Push

Traditionally, software development has been built around a push model of work flow.  That is, as soon as any kind of task is identified, like “add this feature” or “fix that bug”, it’s assigned to an individual.  Each individual has their own personal queue of tasks and they focus only on items in their personal queue.  Some attempt is made to level the queues across the team but as time progresses the queues need to be re-leveled and if they’re not re-leveled frequently enough then some people can become starved for work while others are overloaded.  Over time it becomes very difficult to predict when any given task might be completed because it depends on both who’s queue it’s in and what’s ahead of it in that queue.  While in the work period (sprint, milestone, etc) the question of which stories are done and which are still in progress is really quite non-deterministic.  The only thing that can be said with certainty is that when everyone’s queues are empty, all the stories are done.

Pull

Scrum is fundamentally a pull model.  That is, it’s designed to hold all work in a single common queue and as people become available they pull tasks off the front of the common queue (or as close to the front as possible while allowing for individual expertise, etc.).  In this way there’s no concept of re-leveling and individuals are never overloaded or idle, and tasks get completed as closely to their natural priority order as possible.  It reduces local bottlenecks that might result from individuals getting distracted or sick and makes the overall flow of work more predictable and orderly.  It encourages a style of working where you always have a fairly clear line between “done” stories and “not done” stories so you can easily adjust your plans by simply adding or removing stories from the end of the common queue.

Favor the Pull model

There are advantages and disadvantages to both models, of course, but in general the single common queue strategy tends to be best aligned with the kind of factors we want to optimize for on small teams.  This can be shown by queuing theory and explains why, for example, a single line at a bank or store checkout counter is preferable to multiple lines, one for each clerk.

Agile teams need to choose how they handle tasks in their sprint backlog.  Do they leave them all unassigned and have people claim them one at a time, choosing the highest priority item at the moment that they’re qualified to handle, or do they divide up all the tasks and assign them to individuals during the planning meeting and then everyone works through their individual pile of work during the sprint?

The first option is far less risky, far more adaptable, and far more likely to deliver a cohesive feature set at the end, even if we have to cut certain features.  If everyone is working through a single list from highest priority to lowest, then the most important features come together soonest and there is a clear cut line whenever we need it.

The second option sends people off on their own individual tracks and nobody knows until the end whether everything will come together to give us a working set of features.  We know in which order each individual will complete tasks but we have no idea in which order features will be completed, and that’s the only thing that really matters.  What if someone gets sick?  What happens to their pile of work?  Does it just sit there unattended until they get back?  What if the thing they were working on was on the critical path?  Would anyone else realize it?

Emphasize the team, focus on the product

I’ve heard people object to the pull model saying, “the push approach defines the targets in the beginning.  You know what you need to do.”  That’s true, but it defines individual targets, not product targets.  We effectively tell people, “Ok, here are your ten tasks and if you get these finished, you’re successful.”  The result is that everyone focuses on their ten tasks but no one focuses on how the product is shaping up.  That’s not part of the target.

The idea of agile development is that rather than focusing on individual goals, everyone should be focused on the team goal which is to deliver working software that provides value.  We want all decisions, every day, to be made with that target clearly in mind.  It doesn’t matter what we thought was going to be best two or three weeks ago.  What matters is what’s best today.  We want to tell people, “Here is the feature list and if the team delivers these features, we’re all successful.  Find the one thing that will move us furthest in that direction and do that.”

What about accountability?

I’ve also heard people object to the pull model saying, “If I don’t assign buckets of work to individuals, how will I enforce accountability?”  When you emphasize individual-based metrics, it actively discourages people from collaborating on tasks even when that would be the most efficient strategy from a product-development standpoint.  If I have tasks assigned to me at the beginning of the sprint and I know that I’m going to be evaluated on how many of my tasks get completed, then I’m going to be heads-down on my tasks and I won’t be very inclined to help anyone else in any significant way because it’s a zero-sum game; any time spent helping others directly hurts my own performance.  Yeah, good people will sometimes rise above that and be altruistic anyway, but they’re fighting the system to do so.

Accountability is an important concept, certainly, but it should start at a team level and focus on the product.  You get what you measure.  If you measure working software, you get working software.  If you measure completion of individual tasks, then you get a pile of completed tasks.  Completed tasks don’t deliver value.

If there’s a certain member of the team who’s not pulling his or her weight, don’t worry – everyone will know it pretty quickly.  It’ll be obvious.  There’s nowhere to hide on a small team.  Individual performance issues are best handled out-of-band, not as part of the core development process.

The concept of team ownership over individual ownership and the “just in time” assignment of tasks is a pretty important concept and it’s one of the core principles of Scrum.  It’s pretty much the point of the daily stand-up meeting.  Without it, I’m not sure you can even call what you’re doing “Scrum”.