Technical Rabbit Holes …

Last time we had a good discussion about iterations and how there's a potential for dead-air and confusion near the end of a time-box. Tricky problem with lots of nuances. But there's another place where we can get fooled by the perceived safety of time boxes and iterations. Complex code.

Most interesting software is complex. You're not really treading the high-wire when you're cookie cutting variations on the same app over and over again. The fun stuff comes in when you're solving new or hard problems.

The issue with difficult problems is it's easy to go down technical rabbit holes. To me, technical rabbit holes are when there's a gap between the what the developer is working on and the understanding of it by the customer. It's not that developer is doing something dishonest and obscuring their work. It's not that the customer is unable to understand the problem. The problem is trust. There's too much of it. The developer has just enough rope to potentially hang his part of the project and the customer is willing to let it happen.

And, I know I like to pick on Scrum, but this is not Agile's fault. Agile does a great job of forcing small units of work to be produced and highlights customer interaction and feedback. Agile solved the old problem of “Going Dark” as Steve McConnell correctly describes it: The customer anxiously waiting at the end of a pipe for a product to drop out.

So, then, how can a technical rabbit hole occur?

Well, we know what a good project should look like. We may go off course a little from iteration to iteration, but the customer is there to keep us on track and reach the goal.

When there is no clearly accessible customer or a poor customer proxy we know it's easy to go off course for real. We keep hitting our iterations, we keep delivering code, but it's not the right code (it's not solving the customers problem.)

Dev shops often forget that there are two aspects of writing software: Research and Development. Too often these functions are rolled together into just “Development” and sadly the total becomes less than the sum of the parts. The customer needs to understand whether the developer is doing R or D.

In the above illustration, the Y axis is the customers understanding of what the developer is doing. Green = “I see what's happening.” Red = “I don't really understand, but you say it's important.” You can see that for nearly two iterations, the customer wasn't really comfortable in what the developer was doing. Two iterations. Probably in the best case, that's four weeks of time. That's a long time.

Can you see how this could go bad?

How many iterations do you have to go before the customer finally says “Stop!”?

And, again, it's not that the developer is doing something bad. He's not being tricky or deceitful. He's mostly likely really chasing down some complex code. The problem is he's doing in a manner that abandons the customer. Agile development means developing code together with the customer. It's not Us vs. Them. The customer put too much trust in the developer.

Many times these things work out. But there are times when they don't, when the rabbit hole goes too deep and the developer gets lost. Now we've just spent a lot of time and money and we aren't any closer to the goal for it.

What can we do to identify and prevent technical rabbit holes?

Get a Second Opinion

As with any possible fatal disease, you should get a second opinion. Bring in another developer or two to bounce your approach off. Go deep. Talk about all the nitty gritty issues you foresee. Make them own the problem as much as you do. Ask questions, do they understand your vision? Do they understand your development plan? Do they have any concerns about your approach?

It's these people that are going to help you stay in the light. Perhaps they can articulate your plan to the customer better than you can? Perhaps they can see other ways to deliver on your vision without requiring the customer to put blind faith in you?

Never swim alone in deep water.

Incremental Completeness

Technical rabbit holes can easily occur in any project with some degree of technical complexity. What we should be striving for is Incremental Completeness. Delivering small chunks of working code (with tests) to trunk that, not only help the developer with the R part of Research and Development but, show the customer that we understand where the destination is.

Every iteration, try to put something in the code drop that the customer can see the results of. Perhaps even a simple command line tool the user can run on the release to play with the feature in some fashion (the customer is getting a working build every iteration, right?)

Which gets us to our next point ...

Educate the Customer

Don't assume the customer knows anything about what you're building. They may be able to spout off all kinds of technical mumbo jumbo, but is it code? If they don't understand the piece your building, how can they expect to confirm they have it at the end of the iteration?

Use analogies for the data structures you're building. Try to put it in terms they'll understand. Draw pictures. Share the whiteboard. Explain the complexity.

You may not get it at the first sitting and don't try to force it in a single go. You're telling them a story. Put it in their terms. You're bringing them on a journey, take your time. Just make sure they're not just nodding mindlessly at you.

You're not trying to turn them into computer scientists, but you want to get to the point where the customer can call YAGNI. You want them to tighten the requirements to the point where they can say “we're not going to hit that condition frequently enough … so just print an error message.”

Always Commit

This probably goes without saying, assuming you're using a modern day revision control system, but you have to commit and push your code. Do it frequently. Ideally, no later than every 3 days. Get others to look at your branch. Keep it in the light. Let other developers see what you've been up to. Tell them the same story you told the customer … what do they think? Do they agree that your description is a fair explanation of what you're actually doing? Do they think you're on course?

Don't lose sight of trunk

Merge. Merge. Merge. Pull from trunk frequently . Keep those tests green. If you're doing something that is going to break trunk, use a scaffolding technique: build around the existing mechanism and provide a means to turn it on and off. Later, once your implementation is complete, you can remove the dead code. But, until then, you still have working code. Each iteration you should be pushing something working (with tests) back into trunk.

Letting trunk get a way from you is a sure way to get lost in a technical rabbit hole. If you're adding new code when your tests don't run, you're in trouble. Do you remember what you changed that broke the tests? Are you getting closer or further away from them working again?

As a developer, it's probably your greatest risk.

Mr. Customer: Share the vision

This one is for the customer ... it's ok to sound like a broken record. Keep talking about what the final product will look like. Give lots of detail. Make sure everyone knows where they are going. Make sure everyone knows what's important and what's not. Help the development team understand your business, who your customers are, what they want to experience from this software.

If you keep everyone focused on the goal (and the deadline), it's harder to wander off course. You want the developer to say “You know, I'd like to do this differently, but there won't be time right now. Let me find an intermediate step.”

TL;DR

Technical Rabbit-Holes can be prevented with Eyeballs and Education. Show people what your doing and explain how you're doing it. Don't let too much trust get the better of you.

Good luck … and keep your head above ground!