Change is hard. If we know that about 80% of organisational change programs fail, then it’s easy to appreciate just how hard. Why is that? And, more importantly, what can we do to make it easier?
Recently, I saw a tweet come by from Alan Shalloway. He wrote that, back in the days, people were saying: Waterfall (though they didn’t call it that, back then, I think) is fine, people are just not doing it right. All we need is apply enough discipline, and a little common sense, and it will work perfectly! I think Alan was commenting on an often heard sound in the Agile community about failing Scrum adoptions: You’re just not doing it right! You need to have more discipline
This is, actually, a valid comment. In fact, both views are valid. On the one hand, just saying that people are not doing it right is not very helpful. Saying you need to be more disciplined is certainly not helpful. (Just look at the success rate of weight loss programs (or abstinence programs against teen pregnancies). Change is hard, because it requires discipline. Any process requires discipline. The best way to ensure a process is successfully adopted is to make sure the process supports discipline. This is, again, hard.
This post talks about discipling. About how it can be supported by your process. About how it’s often not supported for managers in Agile organisations, and then of course how to ensure that management does get that kind of support in their work.
In Support Of Discipline
Take a look at Scrum, throw in XP for good measure, and let’s have a look at what kind of discipline we need to have, and how the process does (or does not!) support that discipline.
Agile Feedback Loops
The eXtreme Programming practices often generate a lot of resistence. Programmers were, and are, very hesitant in trying them out, and some require quite a bit of practice to do well. Still, most of these practices have gained quite a wide acceptance. Not having unit-testing in place is certainly frowned upon nowadays. It may not be done in every team, but at least they usually feel some embarrassment about that. Lack of Continuous Integration is now an indication that you’re not taking things seriously.
Working structurally using TDD, and Pair Programming, have seen slower adoption.
Extreme Programming Practices from XProgramming.com
If we look at some practices from Scrum, we can see a similar distribution of things that are popularly adopted, and some things that are… less accepted. The Daily Stand-Up, for instance can usually be seen in use, even in teams just getting started with Scrum. Often, so it the Scrum Board. The Planning Meeting comes next, but the Demo/Review and certainly the Retrospective, are much less popular.
Why?
All of these practices require discipline, but some require more discipline than others. What makes something require less discipline?
- It’s easy!
- Quick Feedback: It obviously and quickly shows it’s worth the effort
- Shared Responsibility: The responsibility of being disciplined is shared by a larger group
Let’s see how this works for some of the practices we mentioned above. The Daily Stand-Up, for instance, scores high on all three items. It’s pretty easy to do, you do it together with the whole team, and the increased communication is usually immediately obvious and useful. Same for the Scrum Board.
The Planning Meeting also scores on all three, but scores lower for the first two. It’s not all that easy, as the meetings often are quite long, especially at the start. And though it’s obvious during the planning meeting that there is a lot of value in the increased communication and clarity of purpose, the full effect only becomes apparent during the course of the sprint.
The Demo is also not all that easy, and the full effect of it can take multiple sprints to become apparent. Though quick feedback on the current sprint will help the end-result, and the additional communication with stakeholders will benefit the team in the longer run, these are mostly advantages over the longer term. To exacerbate this effect, responsibility for the demo is often pushed to one member of the team (often the scrum master), which can make the rest of the team give it less attention than is optimal.
Retrospectives are, of course, the epitome of feedback. Or they should be. Often, though, this is one of the less successful practices in teams new to Scrum. The reasons for that are surprisingly unsurprising: there is often no follow-up on items brought forward in the retrospective (feedback on the feedback!), solving issues is not taken up as a responsibility for the whole team, and quite a few issues found are actually hard to fix!
The benefits of Continuous Integration are usually quite quickly visible to a development team. Often, they’re visible even before the CI is in use, as many teams will be suffering from many ‘integration issues’ that come out late in the process. It’s not all that hard to set-up, and though one person can set it up, ’not breaking the build’ is certainly shared by the whole team.
Unit testing can be very hard to get started with, but again the advantages *if* you use it are immediately apparent, and provide value for the whole team. Refactoring. Well, anyone who has refactored a bit of ugly code, can attest how nice it feels to clean things up. In fact, in the case of refactoring the problem is often ensuring that teams don’t drop everything just to go and refactor everything… Still, additional feedback mechanisms, like code statistics using Sonar or similar tools, can help in the adoption of code cleaning.
Pairing shows quick feedback, and is shared by at least one other person, but it’s often very hard, and has to deal with other things: management misconceptions. We’ll talk about that a little later. TDD’s advantages are more subtle than just testing at all. So the benefit is less obvious and quick. It’s also a lot harder, and has no group support. These practices do reinforce each other, testing, TDD, and refactoring are easier to do if done together, pairing.
Feedback within the team
One large part of this is: feedback rules. This is not a surprise for most Agile practitioners, as it’s one of the bases of Agile processes. But it is good to always keep in mind: if you want to achieve change, focus on supporting it with some form of feedback. One form of feedback that is used e.g. Scrum and Kanban, is the visualisation of the work. Use of a task board, or a Kanban board, especially a real, physical one, has a remarkable effect on the way people do their work. It’s still surprising to me how far the effects of this simple device go in changing behaviour in a team.
Seeing how feedback and shared responsibility help in adoption of practices within the team, we could look at the various team practices and find ways to increase adoption by increasing the level of feedback, or sharing the responsibility.
The Daily Stand-Up can be improved by emphasising shared responsibility: rotating the chore of updating the burn-down, ensuring that there’s not a reporting-to-scrum-master feel by passing around a token.
The Planning Meeting could be made easier by using something different from Planning Poker if many stories need to be estimated, or this estimation could be done in separate ‘Grooming’ sessions. Feedback could be earlier by ensuring Acceptance Criteria are defined during the planning meeting (or before), so we get feedback for every story as soon as it gets picked up. Or we can stop putting hourly estimates on tasks to make the meeting go by quicker.
Retrospectives could be improved by creating a Big Visual Improvement backlog, and sticking it to the wall in the team room. And by taking the top item(s) from that backlog into the next sprint as work for the whole team to do. If it’s a backlog, we might as well start splitting those improvements up into smaller steps, to see if we can get results sooner.
Agile management practices
Probably the most frequently sited reason for failure of Agile initiatives, is lack of support from management. This means that an Agile process requires changes in behaviour of management. Since we’ve just seen that such changes in behaviour require discipline, we should have a look at how management is supported in that changed behaviour by feedback and sharing of responsibility.
First though, it might be good to inventory what the recommended practices for Agile Managers are. As we’ve seen, Scrum and XP provide enough guidance for the work within the team. What behaviour outside the team should they encourage? There’s already a lot that’s been written about this subject. This article by Lyssa Adkins and Michael Spayd , for instance, gives a high-level overview of responsibilities for a manager in an Agile environment. And Jurgen Apello has written a great book about the subject. For the purposes of this article, I’ll just pick three practices that are fairly concrete, and that I find particularly useful.
Focus on quality: As was also determined to be the subject requiring the most attention at the 2011 reunion of the writing of the Agile Manifesto, technical excellence is a requirement for any team that want to be Agile (or just be delivering software…) for the long term. Any manager that is dealing with an Agile team should keep stressing the importance of quality in all its forms above speed. If you go for quality first, speed will follow. And the inverse is also true: if you don’t keep quality high, slowing down until nothing can get done is assured.
Appreciate mistakes: Agile doesn’t work without feedback, but feedback is pretty useless without experimentation. If you want to improve, you need to try new things all the time. A culture that makes a big issue of mistakes, and focuses on blame will smother an Agile approach very quickly.
Fix impediments: The best way to make a team feel their own (and their works) importance is to take anything they consider getting in the way of that very seriously. Making the prioritised list of impediments that you as a manager are working on visible, and showing (and regularly reporting) on their progress is a great way of doing that.
Note that I’ve not talked about stakeholder management, portfolio management, delivering projects to planning, or negotiating with customers. These are important, but are more in the area of the Product Owner. The same principles should apply there, though.
Let’s see how these things rate on ease, speed of feedback, and shared responsibility.
Focus on Quality. Though stressing the importance of quality to a team is not all that difficult, I’ve noticed that it can be quite difficult to get the team to accept that message. Sticking to it, and acting accordingly in the presence of outside pressures can be hard to do. Feedback will not be quick. Though there can be improvements noticeable within the team, from the outside a manager will have to wait on feedback from other parties (integration testing, customer support, customers directly, etc.) If the Agile transition is a broad organisational initiative, then the manager can find support and shared responsibility from his peers. If that’s not the case, the pressure from those peers could be in quite a different direction…
Appreciate mistakes. Again, this practice is one in which gratification is delayed. Some experiments will succeed, some will fail. The feedback from the successful ones will have to be enough to sustain the effort. The same remarks as above can be given with regards to the peer support. Support from within a team, though less helpful than from peers, can be a positive influence.
Fix impediments. The type of impediments that end on a manager’s desk are not usually the easy-to-fix ones. Still, the gratification of removing problems, make this a practice well supported by quick feedback. And there is usually both gratitude from the team and respect for action from peers.
We can see that these management practices are much less supported by group responsibility and quick feedback loops. This is one of the reasons why managers often are the place where Agile transitions run into problems. Not because of lack of will (we all lack sufficient willpower:-), but because any change requires discipline, and discipline needs to be supported by the process
Management feedback
If we see that some important practices for managers are not sufficiently supported by our process, then the obvious question is going to be: how do we create that support?
We’ve identified two crucial aspects of supporting the discipline of change: early feedback, and shared responsibility. Both of those are not natural fits in the world of management. Managers usually do not work as part of a closely knit team. They may be part of a management team, but the frequency and intensity of communication is mostly too low to have a big impact. Managers are also expected to think of the longer term. And they should! But this does make any change more difficult to sustain, since the feedback on whether the change is successful is too far off.
By the way, if that feedback is that slow, the risk is also that change that is not successful is kept for too long. That might be even worse…
When we talk about scaling Agile, we very quickly start talking about things such as the (much debated) Scrum of Scrums, ‘higher level’ or ‘integration’ stand-up meetings, Communities of Practice, etc. Those are good ideas, but are mostly seen as instruments to scale scrum to larger projects, and align multiple teams to project goals (and each other). These practices help keep transparency over larger groups, but also through hierarchical lines. They help alignment between teams, by keeping other teams up-to-date of progress and issues, and help arrange communication on specialist areas of interest.
What I’m discussing in this post seems to indicate that those kind of practices are not just important in the scenario of a large project scaled over multiple teams, but are just as important for team working separately and their surrounding context.
So what can we recommend as ideas to help managers get enough feedback and support to sustain an Agile change initiative?
Work in a team. Managers need the support and encouragement (and social pressure…) that comes of working in a team context as much as anyone. This can partly be achieved by working more closely together within a management team. I’ve had some success with a whole C-level management team working as a Scrum team, for instance.
Be close to the work. At the same time, managers should be more directly following the work on the development teams. Note that I said ‘following’. We do not want to endanger the teams self-organization. I think a manager should be welcome at any team stand-up, and should also be able to speak at one. But I do recognise that there are many situations where this has led to problems, such as undue pressure being put on a team. A Scrum of Scrums, of multi-level stand-up approach can be very effective here. Even if there’s only one team, having someone from the team talk to management one level up daily can be very effective.
Visualise management tasks. Managers can not only show support for the transition in this way, they can immediately profit from it themselves. Being very visible with an impediment backlog is a big help, both to get the impediments fixed, and showing that they’re not forgotten. Starting to use a task board (or Kanban, for the more advanced situations) for the management team work is highly effective. And if A3 sheets are being put on the wall to do analysis of issues and improvement experiments, you’re living the Agile dream…
Visualise management metrics. Metrics are always a tricky subject. They can be very useful, are necessary, but can also tempt you to manage them directly (Goodhart’s Law). Still, some metrics are important to managers, and those should be made important to the teams they manage. Visualising is a good instrument to help with that. For some ideas read Esther Derby’s post on useful metrics for Agile teams. Another aspect of management, employee satisfaction, could be continuously visualised with Henrik Kniberg’s Happiness Matrix, or Jim Benson’s refinement of that.