Monday, 7 September 2009

Moving beyond the Scrum

I used to be a firm supporter of Scrum process. I was deeply convinced that Scrum's approach makes total sense and is the right one. My impressionable attitude has changed with time. Now I see that "pure" Scrum is a quite rigid process which not always fits the best our specific type of work. What is exactly on my mind?


Scrum is stopping us from higher productivity



Sprint length is derived based on project's life expectancy. Having at lest 4-6 sprints within the project is an absolute minimum. Usually we are choosing from two options: one or two weeks for sprint. Unfortunately there are some issues with both options:
  • One week sprint gives us only 4 days of development so one day is "wasted" on Scrum related activities. 20% of our time, it's quite a lot.
  • With two week sprints there is still one day devoted for demo, planning session, retrospective etc. but ratio here is 9 to 1. It's a significant improvement but in the same time we have to deal with new issues which are more evident for two weeks sprint.
    • Unchangeable sprint plan - sometimes it happens that somewhere in the mid-sprint all stories are done or blocked by some external dependency with which we can't do anything. Because we can't add new stories in the middle of sprint and we can't proceed with blocked stories we are forced to fill our time with "other stuff".
    • Another absolutely normal thing is that some new, very important and extra urgent things need to be done ASAP. We can moan that things like that shouldn't happen but in reality they happen so we have to be prepared for it! Scrum's answer to situation like this is "we will take care of that in next sprint as current one is already in progress" or "we can stop this sprint, run planning session again and incorporate this very urgent story into sprint plan". Both options are not good enough, replacing one story with another of similar size in the mid-sprint doesn't hurt the team and in the same time can address important requirement. Scrum is not very flexible here.

Even though there are some issues with Scrum I don't think that we should look for some revolutionary changes. This process works, we just need to tweak it in order to make it more suitable for our projects. Here are a few ideas to consider:
  • Do we need a sprint planning sessions? For sure we need prioritized product backlog to know which stories we should work on in the first place. On planning session we will either overestimate (stories which are not done by the end of sprint will be moved to the following one) or underestimate (and then we need to add some stories in the middle of sprint), priorities may change (stories swapping) therefore plan is very likely to change. Maybe it's better to simply use product backlog as a queue and finish as many stories as we can during the sprint?
  • Retrospective meetings - they are very beneficial when project starts but when it's closing to the deadline they are getting less and less valuable. Maybe it should be team's decision if it makes sense to have such a meeting? If something is going wrong and it's obvious to everyone why should we wait till the end of sprint with retrospective?
  • Demo - usually we want to have implemented functionality available to the client as soon as possible so maybe instead of delivering new stories after one big demo we can have number of small demos and this way be more responsive?

Probably the most important thing which I have learned from Scrum is that we should keep looking for things which can be improved and experiment with the process. So if answer to some of above questions is positive then maybe it's time to move towards more lean processes? Scrum-ban? Kanban?


Last words ...



I have been using Scrum with different deviations for about 3 years now and I still think that Scrum has a lot to offer. The thing is that it doesn't have to fit perfectly for all types of projects. I think I can say that I'm still a firm supporter but not so impressionable any more.

It seems that more and more teams is already moving along this path:

8 comments:

Urs Enzler said...

I don't agree with your arguments. here's why:

>> Now I see that "pure" Scrum is a quite rigid process <<

Scrum is a framework and can therefore not be applied "pure". You have to add stuff to meet your needs.

>> Sprint length is derived based on project's life expectancy <<

I agree that you need a long enough project to use Scrum efficiently.

>> One week sprint gives us only 4 days of development so one day is "wasted" on Scrum related activities. <<

wasted? This time is not wasted at all. This time is an investment in the understanding of the team about the domain. In my opinion, this is the number one success factor: understanding the domain.

>> Unchangeable sprint plan <<

Needed by a high performing team to get stability in the things it develops. Especially important if the team is 7 - 9 people and development takes > 10 sprints.

>> all stories are done or blocked by some external dependency <<

indicates that sprint preparation was not good enough. A sprint contains only stories that can be implemented and all dependencies are resolved. For unresolved dependencies, pre-conditions have to be established first.

>> Another absolutely normal thing is that some new, very important and extra urgent things need to be done ASAP. <<

indicates that priorities were not set reasonably. Two weeks is normally not very long in business time. And testing has to be aligned, too. As you explain,Scrum handles this case with a Sprint abort. Anyway, if this happens a lot then think chaos is ruling you. And the Team (incl. PO) can always decide to skip some stories in favor of another), but think about ROI and the lost performance in this scenario. Scrum is only as inflexible as its team members are.

>> Do we need a sprint planning sessions? <<

Absolutely! Align team understanding. But keep meetings as short as possible. Discuss domain in whole team and not part for part afterward with a lot of repetition and possible misinformation of parts of the team.

>> Retrospective meetings - they are very beneficial when project starts but when it's closing to the deadline they are getting less and less valuable. <<

Continuous improvement is important. People, tools, technology and environment change, therefore retrospectives are needed always. We do retrospectives in our project for about 2 years now. And they still surface important things. But keep in short and focused, of course.

>> Demo - usually we want to have implemented functionality available to the client as soon as possible so maybe instead of delivering new stories after one big demo we can have number of small demos and this way be more responsive? <<

This would be nice, but the really important stakeholders are mostly hard to get to a demo. Therefore, we are happy when they show up biweekly (all of them) and we can show them what went on. It would be impossible to have several small demos throughout the Sprint.

>> it's time to move towards more lean processes? <<

Very true, but don't change the basic things that make Scrum work.

Just my thoughts though :-)

Cheers,
Urs

Marek Blotny said...

Hi Urs,

Thanks for your comments. I even agree with most of them :) In ideal world probably with all of them.

The thing is that are differences between idealistic world and our specific projects. I'm trying to adopt process to the reality to get the best results.

What is so specific about our projects?
- projects are quite short (2-4 months)
- we are collaborating with external companies and those are external dependencies which are not always predictable :)
- tight deadlines - quite often we don't have a luxury of postponing stories because of unresolved dependencies, sometimes we have to be trying to sort everything out during the sprint.


>>> In my opinion, this is the number one success factor: understanding the domain. <<<

That's absolutely true, I agree that team should get as many time/meetings as it's necessary to clearly understand requirements. But I would separate that from sprint planning session where final goal for the team is to make a commitment about the amount of stories which will be delivered. That's what seems to be a waste.

>>> we are happy when they show up biweekly (all of them) and we can show them what went on. <<<

This is another major difference between our clients. As our projects are short there is sometimes a demand for more frequent updates.


>>> We do retrospectives in our project for about 2 years now. And they still surface important things. But keep in short and focused, of course. <<<

I'm far from saying that any team should put this practice aside ... I'm more into having such meetings when there is a need for it.


I think that we using scrum in slightly different scenarios and that's why our approach is different. For sure I agree that changes which teams introduce to the "pure" scrum shouldn't break basic things and that's why I opt for evolution and experimenting.

Cheers, Marek

Urs Enzler said...

Hi Marek

I can see your point. I agree that it is difficult to eliminate waste in Scrum for such short projects.

And as you see, we both adapted Scrum to fit our projects. And that's the important step to take.

Have a nice day,
Urs

Tomek Dabrowski said...

I can't agree that Scrum stops you from higher productivity. Do you really think that one day spent on planning/retrospective/demoing is a waste? Don't you see any advantages if this "waste"? Finally, why have you been using Scrum for 3 years if it constricts you?
My thoughts... Basic idea of Agile is that you need to adjust given process (Scrum, Kanban or whatever it is) to your reality. Of course, you can say that if you are changing the rules of Scrum it means Scrum isn't good enough. But that way of thinking is what it shouldn't be! Scrum offers you lots of good things (Retrospective, Planning, Demo, etc.) and you need to try most of them. Finally, team should decide if given technique works for your project. If not, why you should use it? You said, you are having shorts projects – well it this case I believe you can do project’s retrospective as well and introduce your actions in the new projects.

Marek Blotny said...

>>> Do you really think that one day spent on planning/retrospective/demoing is a waste? <<<

I think it might be a waste, if planning doesn't give us much and we don't have anything interesting to say on retrospective then maybe it would be better to skip some of those?

>>> Don't you see any advantages if this "waste"? <<<

I think that planning, retrospective etc can be extremely useful, but it's simply not necessarily true for all types of projects. There are cases where more lean processes work better (I suppose)


>>> Basic idea of Agile is that you need to adjust given process (Scrum, Kanban or whatever it is) to your reality. <<<

I agree in 100%, but let's say that you are starting with "pure" Scrum and after a few changes your process might not be a Scrum anymore. At least if you try to apply the Nokia Test.

>>> Finally, team should decide if given technique works for your project. <<<

Of course, I'm expressing my view and also I'm just giving a few ideas to consider. Note that I'm for experimenting and trying one idea at the time, not all at once.


>>> Finally, why have you been using Scrum for 3 years if it constricts you? <<<

I think that Scrum is a great process, I have learned a lot during those 3 years. I simply think with more knowledge and experience you can be more selective in order to get process more suitable for you rather then doing everything by the book.

Stephan.Schmidt said...

"Because we can't add new stories in the middle of sprint and we can't proceed with blocked stories we are forced to fill our time with "other stuff".

In the past we've swapped stories that are blocked out, and non-blocked in. Blocked ones went into the next sprint.

Cheers
Stephan

PS: Thanks for linking

Pawel Brodzinski said...

If it is time-boxing which hurts you at the moment, you should be happy with Kanban. Actually you may loosen definition of iteration in your current process, keep the rest which, as appears, works fine and you're almost there.

Kanban board would be a tool which should help you to keep the process under control and deliver you some data needed for reasonable estimation. The rest can remain unchanged.

And I wouldn't really care if it's called Scrum, ScrumBut, ScrumBan or Kanban. As long as it works for your team don't let anyone to destroy it just because it doesn't suit their definition of the label.

By the way: you may be interested in my series of posts about implementing Kanban in our team (
http://blog.brodzinski.com/2009/10/kanban-story.html).

Marek Blotny said...

Hi Pawel,

Thanks for sharing your point of view. For sure I will take a look on your series.

By the way, I have already seen your presentation about Kanban on Agile conference in Krakow so I'm already sold to Kanban :)