How Do I Move to Agile?
It would be ironic if the prescription for migrating to Agile consisted of a series of steps that could be done in a predictable sequence. Then you'd be able to happily Waterfall-plan your Agile transition! In fact, a successful transition to Agile processes is a journey of discovery. Although you know the direction you need to head, there’s no map. You can learn from the approaches of others, but your team and your organization are different, so what practices works best for your team will likely also be different. You may need to occasionally backtrack out of paths that didn’t work out.
In short, a migration to really successful Agile processes will be structured very much like following Agile itself.
If your team is uncomfortable beginning a journey that is unpredictable and not completely planned out, then perhaps Agile is not for you. And software development is probably not for you either. Perhaps you’d be more comfortable building tract homes.
Instead, hopefully you find this discussion liberating! There’s no “ideal” Agile process that you can get wrong, just a sustained, systematic general effort to try various paths, keeping the team practices that prove most effective, continuously keeping an eye on our lodestar of Lean principles.
Changing the Conversation
In my last post, I discussed the need to change the conversation with our customers away from Waterfall-style requirements. For most teams, their principal obstacle to adopting Agile processes is that their customers are demanding Waterfall-style delivery. Understanding why your customer wants Waterfall processes is critical for successfully transitioning to Agile processes.
The first thing I’d ask any team is whether they have credibility to start having an Agile conversation with customers. In my experience, few teams stuck in a Waterfall ditch have demonstrated team delivery capabilities that would allow the customer to plan any other way.
If your customer resists an Agile approach, I suggest postponing the conversation until your team has met a couple of prerequisites. Some internal self-examination may be in order. The good news for many Waterfall-style teams is that they don’t even need their customer’s acquiescence for the first steps to an Agile transition.
If the experience of your customer is a long release cycle, then all this talk about Agile won’t make any sense but simply add uncertainty about what will be delivered. If your customer does not see regular, small, routine releases of usable features — even if they are waterfall planned — then a conversation about Agile processes will not resonate.
Suppose that your team currently releases new features when they are fully baked, roughly every six months. That means that from your customer’s perspective, the top priority business initiative that he or she is presenting to you today may be a year away from delivery! Now think back to where your business was just a year ago and how much has changed. Do you see a problem?
You need to consistently deploy to production at least monthly, ideally at the end of every sprint (assuming your Agile processes uses sprints). As much as possible, completed stories should be routinely put into production. If you cannot do this, then you have internal bottlenecks that you need to fix. These need to be addressed with automation, refactoring, process changes, etc. Set a release cadence goal, then figure out what needs to be done to meet it. Shorten each upcoming release period until the release cycle is measured in weeks, not months. Until this is fixed, your customer is not really the obstacle to Agile — your existing development, testing, and delivery processes are.
This may fundamentally change how you think about partial feature deployment. In some cases, you can roll out the incomplete functionality to all users (when something is better than nothing), in other cases you may need a strategy to “hide” half-baked features that may be confusing to users. A mature approach would be to create “feature switches” so that select end-users can see and evaluate the in-progress features that remain invisible to others.
Many Scrum teams nominally declare their product “ship-able” at the end of each sprint, without actually shipping. I am personally very skeptical that this approach delivers the same value to customers or the same discipline on the team. If this is a server-based application (web site, service, etc.), then not deploying a “ship-able” release may really be exposing a bottleneck in the release process that should be fixed. I’ll grant that kiosk deployments are trickier because of the WAN overhead; however, even there, the benefits of regular improvements to consumer-facing applications are also the most likely to improve our business.
Overnight, we can’t take a team with a six-month release cadence and expect them to release fortnightly. Start streamlining, automate whatever you can, build a checklist for the rest, and continuously get better. There is no “too soon” or “too often” for well-managed releases.
When our customers see development teams routinely delivering small, regular improvements into Production, they’ll start to have confidence in that cadence. You can continue with (modified) Waterfall-style planning (as necessary), even while changing the release cadence. As customers acclimate to the delivery frequency and the smaller planning window, their need to “control” the process may be lessened and they may be more open to a fully Agile approach. Then, planning six-months to a year out will seem obviously unnecessary. In this context, changing the conversation to Agile terms will be far more natural.
In the days before rail or paved roads, you could only guess how long it would take you to get from, say, Chicago to Seattle. However, as you encountered Independence Rock, Fort Laramie, or Fort Boise, or other milestones, you’d have a pretty good idea of the state of your progress. And as you came closer to the journey’s end, you’d have a progressively clearer idea of when you’d finally arrive. Likewise, with Agile projects, in the beginning you can only guess when everything will be complete, but as you burn through the tasks and stories, customers can see progress and the ultimate delivery date and scope will gradually come into focus. That is, I should say: they can see all this if you are reporting it to them!
Agile is about transparency and communication, so reporting is critical. There are two aspects of this: reporting that helps customers make decisions and reporting that drives team accountability. These are related: ultimately, your customer will hold you accountable; what you want is for your accountability to be on terms that are appropriate — your terms. Metrics that are easy to game will simply drive bad behavior.
For team accountability, I encourage four metrics:
- Cycle Time: once a story has been greenlit/prioritized, how long does it take to be released into production?
- Bug Count: is our technical debt growing or shrinking? (Obviously, you might expect this to go up after certain types of reviews, but it shouldn’t after every iteration)
- Code coverage: are we checking in code without tests?
- Build quality: are check-ins causing regressions?
Each one of these reports should be trend based, rather than absolute. The team should be showing improvement on each of these, but setting a specific numeric target as “good” is counterproductive. If you have only 6% code coverage, a trend to 7% is positive and a trend to 5% shows a problem. The trend is all that’s important. These metrics should not be used to compare teams.
As it happens, each of these reports exists as part of the TFS reporting environment — except Cycle Time. A real Cycle Time report will require some integration with Octopus that doesn’t exist yet. Notice that a focus on “commitments”, “schedule”, “estimates” are not a healthy part of team accountability.
The other critical customer reports are the “Burndown” charts (or the related “Burn-up” chart). If your team does Sprints, a Sprint Burndown chart shows the estimated work that has been scheduled for this sprint. As the work is completed, the remaining work value drops. By following the trend-line you can estimate the how much of the scheduled work will be complete by the end of the sprint.
The point of the chart is to highlight progress on work that the customer has prioritized for this sprint. It is not the goal of the sprint to burn down to zero! Something is wrong with a team that always completes exactly what they estimate. The Burndown Chart will highlight how good your estimates are, but if your estimates aren’t accurate, Lean principles suggest that you not spend any significant time improving them. Just try to do better at you next sprint planning meeting.
If your team uses Sprints, the Sprint Burndown will give your customer a straightforward visualization of your team’s intra-sprint progress. With only a small amount of experience, the customer can make their own estimates about what work will be delivered at sprint’s end. Non-sprinting teams can offer similar insight with the Continuous Flow diagram.
A related report is the Release Burndown chart. This shows the progress of the team as they complete the stories the comprise the entire release. Although you can’t commit to a specific release date, you can give your customer the best information you have about when the release is likely. The Release Burndown tries to show this.
The out-of-the-box Release Burndown report in TFS assumes that you’ve defined a Release as parent of your Sprint iterations. That may not necessarily be how you’ve defined your iterations (assuming your teams uses Sprints). Personally, I’d prefer a “Feature” or “Epic” Burndown chart which would give my customers visibility to progress implementing a Feature or completing an Epic. This would be useful because teams might work on multiple features and it would be useful to see their likely delivery, especially when teams are shifting work around. It would also be more useful to Kanban teams. Today that report doesn’t exist, but the robust reporting infrastructure behind TFS would allow you to create it.
If you are stuck in a Waterfall-style conversation with your customers, make sure that you have these prerequisites met. You can change the conversation only if you have credibility as you describe an Agile-style delivery.
One question that was raised as a result of this series was “So if there is a fixed delivery date, is a true Agile methodology automatically ruled out?”
Perhaps. You’ve all seen the Price/Quality/Speed list, that suggests you “Pick two”. In the absence of any other variable, the two ways of meeting a fixed date are by increasing the cost (say, by building in extra margins in all your estimates) or reducing the quality. With a mature customer relationship, the customer should be focused on the upcoming iterations, just like the team is. Given the ability to deliver often, even with changing priorities, the team’s customer have control, flexibility, and agility. However, the real world doesn’t always work that way.
Fortunately, real world projects also tend to have “nice to have” features or lower priority stories. If you know exactly when you need to release, then an Agile team can’t possibly know what will be included. Make that clear to your customers. Give them a Burndown so they can get a rough idea which stories will make the release. Do the most important work first, then complete as much as you have of the rest.
You see this approach all the time as large software vendors prepare for RTM; you hear PMs saying that they’re not sure that this or that much-requested feature will make it.
On the other hand, if the delivery date is fixed and the final requirements are immutable, then Agile planning is not an obvious fit. In that instance, the best argument for Agile planning is that your team is structured around Agile and you don’t want to abandon all of your current processes just for that one release. And, hopefully, by having the customer participate in Agile delivery, subsequent projects will be better adapted to software delivery.
There has been a lot of talk in the series about Lean principles. Most people have heard of Lean, but probably not in enough detail to justify using it to make decisions. Lean’s origins in manufacturing have required some adaptation to software development. You can find lots more information about this online.