If you know me, you know I'm not a big fan of Scrum. I don't like the hand-waving concept of Velocity and I don't like the fact that it places a priority on process over code. XP, I feel, is a much more important for software development. Agile (aka customer responsiveness) can be achieved without the dogma. But that's not the point of this post. In this post I want to question the value of timeboxing and the benefits that modern revision control systems grant developers. So, stick with it, I spend some time setting it up before getting into the meat of the post.
Back in the 90's I was using Spiral Development Model on a project. Spiral is Waterfall with shorter, fixed-length, time spans between milestones. In XP, this idea was formalized as Iterations by dropping the Design phase and using automated testing to replace the long, drawn-out, testing phase.
The rationale behind iterations and Spiral was simple: Estimating is hard. The further out you look, the greater your error rate.
Spiral's insight was to reduce risk by keeping the milestones close together (and ensuring that each milestone is a shippable product).
Scrum expanded on this idea and permitted the functionality of the project to change from iteration to iteration. It does this by bringing the customer back into the picture each iteration for sign off and review of “what's most important now”. All good stuff.
At the start of the project, the customer has a general idea of what they want from the development effort. Working with the team, these requirements manifest themselves as Stories (aka Requirements ... friggin' Scum wants to rename everything)
Now the developers enter the picture and give some guesses about how difficult each story is. This is usually done as T-Shirt Sizes (Small, Medium, Large) or some other metric. In a perfect world, these sizings are not equated to Time ... they are relative efforts and no planning occurs more than one iteration out. But in reality, the customer needs to have some idea for when to expect their product. It could be as coarse a resolution as 1st Half of Next Year or 4th Calendar Quarter, etc.
So ... we're back to using Time again. T-Shirt sizes get equated to some unit of time (let's say, Small = 1 day, Medium = 2 days, Large = 3 days). Iteration sizes are set into something manageable such as 2-3 weeks each.
Finally, given the number of developers it's a pretty easy exercise to give a Wild Assed Guess (WAG) as to when the product will ship. If everyone is happy, the dance can begin with the first iteration.
When the developers get into the first iteration they start coding on the most important stories (as selected by the customer). Things are wonderful. But, when they get into the second iteration some things may have changed. The priority of the stories may have changed, the first iteration dev effort may have uncovered new stories or perhaps, there is already technical debt that needs to be addressed. Regardless, the original WAG deadline is likely changing (for better or worse).
The reality is that time-boxing the iteration introduces waste near the deadline. Let's say I'm 2 weeks into a 3 week iteration and I've finished all my work early, what should I do? Should I start working on another story from the next iteration (which includes bug fixes)? Should I perform some other non-development task such as manual testing or documentation? Should I refactor?
Surprisingly, the Source Revision Control System (RCS) that you are using can have a significant impact on your decision. Personally, I think Distributed Revision Control Systems (DRCS) such as git, bazaar or mercurial are the most significant change in software engineering within that last 10 years. Not because having a non-central repository is so revolutionary, but because branching and merging has become such a low-cost operation that the speed of software development has increased dramatically. Development shops that utilize these tools can see some big improvements in how they deal with the “Rough Edge” at the end of the sprint.
Let's compare dev shops that use a RCS that supports fast/low-cost branching and merging to those at don't.
As I mentioned above, having rigid deadlines means potentially having dead time on your hands.
You either have to find small, low-risk stories that you can take on within the remaining time or do busy-work to fill the gaps. If your customer is more willing to let the deadline slip to get the functionality they desire, things aren't much better. You're simply moving the dead time out a little further.
But if you're using a RCS that has low cost branches and good merge capabilities, things get much easier because the developers can keep working on upcoming stories without affecting the current iteration.
“But I can branch and merge with svn or
” I hear you say. Yes, you can, but not quickly. When keeping in sync with trunk is expensive, it doesn't happen. When you don't sync with trunk the costs of your merges increase, merges are larger and your unit test maintenance efforts increase. When branching and merging costs are low, it's easy for developers to start and new branch and keep working without upsetting trunk or the deliverable for the current iteration. Look at the NVIE branching model as your reference standard.
So, the million dollar question ... if I can keep working and still ship my promised set of stories as they become available, why time box in the first place?
Pull models such as Kanban don't really rely on timeboxing. Instead, developers pull stories from the backlog when they need to. The customer can re-prioritize the backlog as they desire. Finally, when a feature is completed it's merged into trunk and becomes part of the deliverable. The customer can decide which version of trunk they want to use and don't need to wait for an N week deadline to pass before grabbing the product. Developers are encouraged to keep the Work In Progress (WIP) to a minimum to prevent having lots of half-finished efforts.
We, as developers, can still give estimates on each feature and we can still do our burn up/down charts to track progress, but we don't need to actually block out time frames. Actually what we're doing is taking the idea of iterations to the logical extreme: each story is an iteration.
I can think of only one situation where time boxing is useful and that's in Scrum of Scrums. When the senior managers/customers have to monitor more than one team of developers they need to be able to snapshot the state of progress across teams. The ideal situation however, would be a Scrum of Kanbans:
Senior stakeholders get a top-down view of the state of the development process without slowing down the developers every 2-3 weeks. For them, this is a simple synchronization mechanism for keeping the cats herded.
This gets us back to one last point on estimating. As I've said before, estimating sucks. Developers hate having to break down tasks to such a fine resolution that they can be tracked in 4-8 hr intervals. If I have to think about a problem to 4hr resolution I may as well just code it. Why not 1 hour? Why not task out to every 30 minutes? Simple, it's a case of diminishing returns. The time it takes to document the tasks outweighs the effort itself.
I think a better approach is to not to track tasks at high resolution. Instead keep things in the 1-3 day range with a clear deliverable and simply track the sentiment of the developer towards the story. On the first day of the iteration, the developer should be very positive that they can deliver this story in this iteration (otherwise, why did they sign up for it?) ... but as the iteration progresses their sentiment may change. And it's going to North or South very quickly. “Oh, that's not good”, “This is getting bad.”, “I'm screwed”.
This is what we should be getting from our Standup Meetings. When we report status and obvious impediments to our peers we should also be giving a vibe on our ability to deliver.
This gets ever better for the Senior Managers, not only can they see their Scrum of Kanban stories getting executed, but they can boil up the sentiment from each of the teams to see how the iteration is going.
What do you think? Would your daily development process be better if you didn't have to break down tasks to super-fine resolution? As a manager, could you better estimate ability-to-deliver based on higher-level sentiment vs. tasks completed?
UPDATE: Glen Campbell was nice enough to give his perspective on this post. Jump over there to read his post and the comments to see my response.