Wednesday, April 18, 2007

Understanding Agile, Take 1

To the optimist the glass is half full.
To the pessimist the glass is half-empty.
To the agilist the glass is twice as big as it needs to be....

Tuesday, April 03, 2007

Cruise Control health meter

I use Cruise Control. It’s my automated build weapon of choice. Why? For no other reason that it was the first one that I got my hands on. I (mostly) understand it, trust it and know its quirks and foibles.

It also has a neat feature – it shows project metrics. These are shown as pie and scatter charts displaying successful and unsuccessful builds. Both of these provide useful indicators about the health of a given project.

Taking the scatter chart first. This is a calendar with day on the X axis, and time on the Y axis. A dot is placed on a calendar every time there is a build – red for fail, blue for success. From this you can get a feel for the following:

  • How many checkins are being done per day
  • How successful they are
  • How long a build stays broken
  • Whether the Team is staying unusually late or coming in at weekends

All of these are useful indicators as to how things are going in general. I look for regular checkins during standard working hours and fast recovery from breakage – and I encourage the Team to investigate if this pattern is broken. All fairly straightforward and uncontroversial in the agile world.

The pie chart does something similar – it shows the proportion of broken to successful builds in isolation. Not necessarily as useful as the calendar scatter, or is it?

I have noticed an interesting trend with the pie chart in Cruise Control. At the moment it is nothing more than a gut feel or rule of thumb based on the observation of several teams, and I certainly have no data to back this up….but successful, dynamic teams appear to break somewhere between 15% and 25% of builds.

Shock! Horror! Agile teams break the build!

Well, yes. Breaking stuff unexpectedly is how you learn and improve. It is also the motivation to fix structure and design. So the point I am making is: Yes, teams should expect to break the build once in a while and question if they are not. It would seem that somewhere around 20% is the sweet spot where the Team is behaving optimally. Much more than this and something is going seriously wrong. Less than this and there is either not enough refactoring pressure on the design or the design is almost perfect (I know which of these I would suspect initially).

"Learning is not compulsory" - The Dangers of Faux Agile

I am noticing a worrying trend amongst companies wanting to take advantage of the perceived benefits of the Agile Revolution – specifically the faux-agile project. These are projects that have stamped themselves as “agile” but do not adhere to even the most fundamental good practice. When you scratch the surface they do not continually build and test, nor do they actively refactor bad design out. Unit tests are at best given lip service, and coverage is low. In most cases the only nod towards anything remotely agile is the lack of unnecessary – or indeed any –­­ documentation.

When asked, the excuses for the shortcomings are many and varied – “We know the structure of the software is awful but we don’t have enough time to refactor it right now, but we will get around to it later when we have more time” (yeah, right…), “Our Company culture does not allow the Quality Department to be integrated into development teams”, “We don’t know how to implement the automated build”. The list of excuses given by faux-agile apologists to explain away their inability to address core agile principles is endless. And yet when the proverbial hits the fan and the project is not delivering, someone always has the audacity to claim that it was the agile process that failed! In reality the Team was using that time-honoured (but dishonourable) process called hacking and hiding it behind the Agile flag of convenience. I am sure I don’t need to expand on the potential damage that this can do the the Agile cause.

So why are these pseudo-agile projects becoming more common? First of all I am convinced that it is not deliberate dishonesty – all the members of faux-agile teams I have met and worked with are genuine, honest software developers. Some have even convinced themselves that they are following the Agile Manifesto.

So if it is not deliberate subterfuge, what drives these behaviours?

My opinion is that it derives from the interaction of several issues.

  1. The cost of initial implementation of agile. Kicking off (or converting) a project to ‘agile’ requires some up-front investment. Setting up an automated build machine, defining an acceptance framework, educating the Team in agile process and so on.
  2. Management cynicism. I believe that Management in general is still suspicious of agile methods because they transfer a significant amount of power from the manager to the development team. Add to this any significant cost of implementation (including development down-time), and a pinch of failed previous faux-agile projects and suddenly the tension rises – “Why should I, a traditional manager, risk losing a large amount of budget on an experiment like this? You can do it as long as I don’t see any downtime or schedule slip”.
  3. Lack of understanding of the “Agile Way”. When a Team does not fully understand the holistic nature of an agile process, they will be tempted to cut corners and only implement specific parts, leaving the rest “until later”. This is especially true if they are under pressure to put it into place with no downtime or schedule slip. This is a direct consequence of skimping on Agile training – buying the Team a copy of Extreme Programming Explained does not usually magically make the Team agile…

And so the Team begins to stray towards the dreaded Whirlpool of Doom….not all the framework is in place so the Team works a bit harder to make the process work which means they don’t have time to fix the framework which means they have to work a bit harder which means they have even less time to fix the framework….

So how can we fix this? I would suggest:

  1. Companies must begin to recognise that “agile” is not a silver bullet. It is an alternative way of developing software, and as such it does have a high risk of failure if not installed correctly. As such it will have a cost of implementation. This will involve proper training of the Team and Management and Customer. It will also involve the setting up of the infrastructure – automated build machine, acceptance test framework and so on. The exact cost depends entirely on how close to the agile ideal you are before you start – in other words how far away you are from your destination.
  2. As with any major change, give the Team space to make initial mistakes. They will be better and stronger for it. Give the feedback process a chance to work. Enable the Team. Let the Team learn.
  3. Don’t be afraid to get help! This is not simply a sales pitch for the Coaches and Mentors out there (myself included), it is an observation from the trenches. There are many guys (and girls!) out there who have “been there, done that, got it wrong, then got it right”. Some have decided to share their knowledge to help your Team avoid the mistakes they made initially. The right Coach will set up the Team and leave it in a state where it can be used to seed other teams later, ultimately making your Company self-sufficient in agile training. The short term cost outweighs by far the long term benefit of getting it right.

And what if your Company is not prepared to adapt and implement agile process in a responsible manner, instead insisting on perpetuating the faux-agile anti-pattern? The best advice I can give is to ponder a quote from Lord Deming – “
Learning is not compulsory... neither is survival”.