Don't shave that yak!

(If anyone would like to claim this is their work, I am more than happy to recognise the fact - credit where credit's due. Just drop me a line)

Estimation is dead! Long live estimates!

Currently it seems trendy to say that your team no longer estimates. Or that estimating work is pointless (pointless? Geddit? oh never mind....). Hell, even I have said similar on Twitter.

So estimation is finally dead. Hurrah! No more picking meaningless numbers out of thin air when you really have no idea how long something will take. Finally we are rid of a process that is demonstrably flawed.

Except...is that a twitch I see from the lifeless corpse? I do believe it is!

There will always be times when it is genuinely useful to answer questions such as "Will we be able to show our new product at the April trade show?". Or "Which quarter do we need to start ramping up our marketing for the new feature?". All of which require estimates. Not necessarily to-the-day accurate estimates, but still estimates.

Even teams who claim not to estimate are finding that it is useful to make a judgement call - dare we call it an estimate - on the size of a requirement that they are about to pull into play. It might be to see if it can be done in a reasonable time to satisfy a service level agreement, or it might be to make sure that it is about the same size as other stories that have been played in order to reduce variance and improve flow. But this is still an estimate.

So estimating has taken a beating - and rightly so, having been abused for so long. But what is emerging is a better, leaner form of estimating, supported by real live feedback, that is far more fit for purpose.

An achievement!

Hurrah! I have earned another piece of paper (with distinction, no less :-) )! This time for completing the excellent Functional Programming with Scala course run by Martin Odersky and Coursera. I have touched on functional programming before, back in the days when I used to write XSLT for a mashup engine, but this course really helped open my eyes to what the underlying principles are. Not to mention forcing me to reassess the level of test-driven you apply to the new techniques.

Now the journey really begins!

Dijkstra, tools and version control systems

“The tools we use have a profound (and devious!) influence on our thinking habits, and, therefore, on our thinking abilities.”, Edsger W.Dijkstra

...or in other words...

"If you already have a hammer, it is very tempting to treat everything as a nail"

My views on using electronic tracking tools are already fairly well known. But another area that creates a huge impact is the choice of version control system. Using the right software will help your project deliver quickly and efficiently. Choose the wrong one, and the project will be nobbled by unnecessary, damaging and sometimes downright dangerous processes and maintenance overheads right from the start. 

In order to avoid the trap highlighted by Dijkstra, it is essential to match the tool to the way you develop and not the other way around. 

Let’s consider a fairly typical, highly productive and successful team. In my experience they tend to follow a fairly well defined pattern of working:

• All developers get a copy of the current mainline code out of the VCS.
• The developers work on the code, making a relatively small changes, including various tests, as they go.
• Everyone regularly checks these changes in to the mainline. This happens at least once per day. Normally checkins are much more often - 5-10 checkins per developer per day is not unusual for a fast-moving team. 
• After each checkin, the mainline code is compiled, both unit and regression/acceptance tested, and packaged. Note that assuming no problems are encountered, there is potentially deployable product each time this happens.

Regular, small checkins to the mainline branch means everyone’s code is integrated many times per day, revealing problems quickly (fail fast). The amount of change between a successful build and a problem is limited, and so makes recovery more manageable; integration issues can usually be resolved within 15 minutes. 

This way of working eliminates branching almost entirely, so avoiding the dreaded Merge Hell. Everyone always works on the mainline branch, no-one uses a separate branch to develop. Changes and bug fixes are always on the mainline, so the software is always going forward using small, manageable integrations. In the rare case that a change needs to be applied urgently to an existing release before the next formal release, then a separate branch might be created to handle this (alternatively a new copy of trunk may be deployed). Releases are, of course, fully tested at a number of levels and accepted before release, so problems should rarely occur. 

Developing this way also has “a profound (and devious) influence” on what our versioning software needs to be able to cope with. Here are just a few I can think of:

• Each automated build needs to be uniquely identifiable with minimal effort so it could be released if needed
• It needs to be easy to rollback individual changes/checkins. Fail fast also implies recover fast.   
• Checkins and checkouts need to be fast and efficient so they are seamless
• Good integration with tools to make its use virtually seamless (developer IDE, continuous integration setup and so on)
• Easy to use. Amongst the “small changes” a developer might need to do is to rearrange the project structure as required to maintain a successful develop-and-build cycle as new requirements become understood, or new components are identified

Compromising any of these would be likely to obstruct the developers as they build software[*]. Worse still, as Dijkstra suggests, the tool will affect the way developers think and work:

• If builds cannot be easily tagged/identified, then they won’t be tagged
• If it is hard to roll back, then developers will not roll back
• If it is slow to check in or check out, then developers will only do these when forced to. Frequent integration becomes less frequent, problems become harder to find
• If it is hard to integrate, build and test, then it won’t happen very often
• If it is hard to reorganise the project, then it won’t be refactored to a better structure

...to counter this, suddenly you have to issue edicts, define rules and issue development documents and ticklists to ensure people work around the bad tools and make sure the “right stuff” is happening. Trust me - I have seen it happen and experienced it first hand; it is a very bad place to be!

Whenever choosing a tool to support your project, bear Edsgar Djikstra’s quote in mind. Make the tool fit your way of working. Don’t let the tail wag the dog.

* I'm looking at you, ClearCase....

A made up conversation[*]

Team: “Coach! Coach! This agile process stuff you sold us doesn’t work!”

Coach: “Did you build working software regularly?”

Team: “No, it was too difficult.”

Coach: “Did you extend your definition of Done as far as you could?”

Team: “No, it was too difficult.”

Coach: “Did you make quality part of your process?”

Team: “No, it was too difficult.”

Coach: “Did you review what you were doing at all?”

Team: “Yes, but we didn’t like what we discovered, so we ignored it.”

Coach: “So are you absolutely sure it was the agile process that caused you to fail?”

Team: “Oh yes, absolutely….”

Coach: "<sigh>"

* honest....

Good, Bad, Puzzling Retrospective

By popular request...here is a blow-by-blow run through of the simple 60 minute 'Good, Bad, Puzzling' retrospective format that I use. Have fun using it. 

Good, Bad, Puzzling Retrospective

Introductions and checkin (10 minutes)

Start off the retrospective by getting people to “check in”. That is, invite them to speak early on, focussing on what is being reviewed. This makes them feel like they are part of the meeting, and implicitly invites them to actively participate.

I normally use something like:

• Describe the last Sprint in 3 words
• If the last Sprint was a Star Trek/Star Wars/South Park character, which one would it be?
• What colour was the last Sprint?

...and so on.... Get creative!

Once everyone has checked in, you need to create a “safe zone” where the Sprint can be discussed without blame. Everyone needs to agree that everyone in the Sprint always acted as best they could under the circumstances at the time, and to accept that anything that went wrong has happened, and cannot be ‘unhappened’. Or in other words, everyone needs reminding that “Hindsight is always 20:20” - you can always look back and say that “if x, y and z hadn’t happened, then something wouldn’t have broken”. I use Norman Kerth’s Prime Directive as a starting point:

“Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.”

Also worth remembering are the Four Freedoms (also from Kerth):

“Everyone has the following freedoms:

• Freedom to recognise and observe what is rather than what others want you to see

• Freedom to ask about puzzles

• Freedom to acknowledge and talk about what's coming up for you

• Freedom to talk about not having any of the above freedoms”

Often improvement is obstructed by a feeling that certain things (problems, behaviours, issues) are off-limits; the Four Freedoms are a great way to remind people that nothing is off-limits as long as it is discussed respectfully.

Brainstorm (10 minutes)

Get everyone to write down their observations and issues from the last Sprint onto Post-Its (one per issue!), and put them onto a board under one of three headings: Good, Bad or Puzzling. If possible, use a colour code - usually green for good, red for bad - this will help identify hotspots later. Also I encourage people to use a medium marker for the sake of readability, and to limit the amount that can be written.

Timebox this to 10 minutes. If everyone appears to have finished before the 10 minutes is up, wait a little longer since someone will almost always remember something. Remember: don’t be afraid of using silence.

Note that it can be worthwhile reminding people to write legibly!

Group (5 minutes)

Get the Team to organise the Post-Its into groups of related issues on the board. Exactly what constitutes a group is up to the Team. Timebox to 5 minutes max. Encourage everyone to participate - do this actually at the board, get everyone moving since it increases their energy for the discussion.

Optionally do this exercise in absolute silence apart from the final minute (to solve any disputes). The idea behind this is that speaking is controlled by the left side of the brain (the logical side), which dominates. Doing this silently is supposed to use more of the right hand brain (the creative side). Does this really work? No idea, but it’s fun....

Note that if you have used colour coding for the issues, specific areas will stand out as being predominantly good, bad or puzzling. Look out for these, and if there is one area that has a large concentration of bad comments, then consider highlighting this, skipping the vote and discussing the issue directly.

Name and Vote (5 minutes)

Once grouped, go through each and give each group a specific name. Identify what links all the issues together.

In order to pick just one topic to discuss, get the Team to do a “3-Dot Vote”. Each team member gets 3 votes to cast in the form of dots against specific issue groups. They can put all 3 against one group, 2 on one, 1 on another, or all three against different groups depending on how strongly they feel about them.

Once everyone has voted, add the totals, and the winning group has the most votes. If there is a tie, then I tend to use a simple show of hands to choose one issue to discuss.

Discuss (25 minutes)

So what we have now identified is one single issue or area that the Team has jointly agreed to discuss how to improve. Sometimes the discussion will happen quite naturally, sometimes you will need to start people off. The best way is usually to focus on what the Team believe is happening - “The Problem”. 

As specific problems emerge, get the Team to consider specific solutions. Don’t lose track of either - I tend to jot them up freeform on a whiteboard. 

As the timebox draws to an end, get the Team to agree on one or two of the solutions that will be adopted as actions in the next Sprint. Don’t allow the Team to overload themselves with change; I generally suggest a maximum of 2 retrospective actions for a two week Sprint.

Wrapup (5 minutes)

Finally, wrap up the meeting. Thank everyone for their time, and summarise the actions that were agreed.

Consider doing a Return On Time Invested (ROTI) exercise to see how people feel about the retrospective, and to provide some feedback on how effectively you ran it. 

For example, “On your way out, vote on a scale of 1-4, where 1 means this retrospective was hugely valuable, and 4 means it was a total waste of time and I may as well have been sitting at my desk working”

Follow Up

Follow up the retrospective with an email stating at very least the agreed actions for the Sprint. 

Other Thoughts

I don’t recommend revisiting the other issue groups identified by the Team at a later date. The retrospective provides a snapshot of what was important to the team at that moment in time, and this will change Sprint by Sprint. If an issue continues to be important, then it will appear again in a later retrospective.

However, if you want to keep track of how retrospective issues change over time, take pictures of the grouped board. You may see recurring issue groups that has been missed by the Team.

Also you may want to take pictures of the notes (problems, solutions) discussed to serve as a reminder. A wiki is a good, informal place to store this information.

Timing

	Timebox	Running  time
Introductions & check in	10	10
Brainstorm	10	20
Group	5	25
Name and Vote	5	30
Discuss	25	55
Wrapup	5	60

I've been nominated for an award!

Well, that’s a nice surprise! I have just received a very nice email telling me that I have been nominated in the Agile Special Recognition Award category at this year’s Agile Awards.