Monday, May 08, 2017

Of Goals, Backlogs and Sprints

Jean-Fran├žois Millet (II) 014
As an experienced developer/coach, once in a while you get asked a truly fantastic question by a client who has actually done her homework, and you're absolutely stumped by it. I love it when this happens since it is a fantastic opportunity to not only learn, but also to teach the client how to solve similar questions themselves next time.

This particular team had been attempting to implement Scrum for some time, with a degree of success. But one thing was bugging them in particular. How to control stories in-sprint. The advice they had been given (and the way I understand Scrum to work) was that sprints are immutable. During the plannig game, the team sets the sprint backlog, populating it with the subset of backlog stories that the team believe they can fit into the timebox. That's it - only a major event causes that to change, in which case the sprint is terminated, replanned and restarted.

So when the Scrum Master asked about the 2016 edition of the Scrum Guide, and why it seemed to contradict that view, I was somewhat surprised. Here it is, in black and white:
As new work is required, the Development Team adds it to the Sprint Backlog. As work is performed or completed, the estimated remaining work is updated. When elements of the plan are deemed unnecessary, they are removed. Only the Development Team can change its Sprint Backlog during a Sprint. The Sprint Backlog is a highly visible, real -time picture of the work that the Development Team plans to accomplish during the Sprint, and it belongs solely to the Development Team.

Woah! Wut? What madness is this? The Scrum Manual contradicting Scrum as I understood it for years? Having battled with seemingly random, moving requirements during my early professional years I can relate to why processes such as Scrum limit the amount of churn. But this paragraph seems to encourage it.

What to do? What could I do to help? I did what any sane person would do in a situation like this, and asked some People Who Know - other coaches in my network and got three different answers back. Hmmm…obviously some room for interpretation here.

So, having scratched my head for a while, it turns out I had missed some nuances. Here's a fourth point of view, hopefully pulling in the salient points from the other three.... The documentation is not giving out an inconsistent message at all. But it is subtle, and open to (bad) interpretation if you don’t pay attention.

Looking at the wording above in isolation was the problem. Reading it in context, a previous paragraph says,
The Sprint Backlog is a forecast by the Development Team about what functionality will be in the next Increment and the work needed to deliver that functionality into a “Done” Increment.
(my emphasis)

So what is being added to/dropped from the sprint by the team mid-sprint is the work needed to achieve the functionality. So it is the functionality (defined in User Stories) that is fixed ("At the end of the sprint, the software wil do x, y and z that it didn't do last week"). Tasks (build a database, define a service etc etc) to get there can vary - which is unsurprising.

I believe one reason I missed this subtlety was because I generally recommend teams do not split stories ("functionality") into tasks since in my experience it encourages not finishing - all the easy things get done, but the software still doesn't work! Also I have found it encourages micro-management, and the use of the severely damaging "Predicted time/Actual time" fields in reporting tools like Jira. Sometimes splitting things out is useful, but usually less so. Most teams don’t miss this micro-planning step, and are very succesful without it.

I think this explains the whole mutable in-sprint work backlog pretty well. Now, what about the Sprint Goal mentioned in the title? It is another much abused Scrum artefact. Most (all?) teams I have worked with have had trouble with it, and in some cases just shoehorned it in retrospectively after selecting what to put into the sprint "because Scrum says so" (<sigh>). Looking deeper into how Scrum sees the Sprint Backlog also clarifies the purpose of the Goal.

The Scrum Manual doesn't have any explicit definition of the Sprint Goal that I can see - but the Goal appears to be there to focus the team on the answer the important question "What is the most important thing this software product needs to do next?" . And this is what links it to Features/Stories.

The Sprint Goal helps define what Stories to play next, what Features to implement next.
This means defining the Sprint Goal needs to come before defining the sprint backlog, not at the end as a retrofitted afterthought. Indeed, this is exactly what the manual says:
Sprint Planning, Topic One: What can be done this Sprint?
The Product Owner discusses the objective that the Sprint should achieve and the Product Backlog items that, if completed in the Sprint, would achieve the Sprint Goal
So the only logical order for the Planning Game has to be:
  1. Define the Sprint Goal (the Product Owner should already know this, or have a pretty good idea of what it should be)
  2. Pick the Sprint Backlog stories that will satisfy the Goal
  3. (Optionally!) break the stories into tasks (“work needed”)
I hope that clears up some confusion out there.


Allan Kelly said...

Great catch Chris! - I was unaware of this erh, subtly (or change?) myself but the fact that it is to subtle, and catches people like you out, shows that Scrum isn't as simple as it seems.

Personally I've never gone in for this "sprint content will not change" approach. I know why its good but I know most teams are not in a position to work like that. My answer (which you probably already know) is Yellow cards "Unplanned by urgent". I've got a full discussion in Xanpan, can I plug that book?

Unlike you I do like to see stories broken down to tasks, to my mind that is a design exercise. Although I have no time whatsoever for "actuals" - or retrospective estimates to give them their correct name.

Chris said...

Thanks for your thoughts, Allan (the pending comment nearly got lost in the system!).

A couple of extra musings on my part:

I think the beauty/problem with Scrum is that it is quite simple. Deliberately so. But in the same way as the game 'Go' is "simple" - the basic rules are easy to learn, but it takes a lifetime to master the tactics. So yes, you can bend the rules, but you first need to understand how far they bend, and when they break (which you do. Others less so)

Regarding breaking stories into tasks, I agree that it is a design exercise. But I have always argued that teams should generally not do design while planning - only the most basic, hand-waving guess of what is needed to deliver the requirement for sizing purposes ("Do a front end, add a database table, glue them together" type tasks), and even then only if if it seems valuable (bearing in mind it will almost certainly be horribly wrong!). Trouble is, any design at the planning stage risks being concreted in too early. Do the detailed design during/after clarifying the requirement (eg using Three Amigos pattern). Et voila! You get the pattern described above! :-)

Finally, I vaguely remember someone pointing out that design/architecture is only ever fixed when the software dies. Was it in one of your articles?