Some organizations practicing agile simply maintain a prioritized backlog and have their teams pick up each item as they have bandwidth. The product owner and the team know the team’s velocity and therefore have an idea of how many items they’ll complete. At Streamline Health we take a more structured approach for two reasons:
- When teams are just forming and adopting scrum, clear goals and commitments make it easier to form disciplined habits. Once this practice is in place, there’s little need to change it.
- Sometimes we must make commitments to clients, so we have to know for sure whether an item will be done in the next sprint.
Therefore, at Streamline, our product owners package each sprint and our teams commit to delivering them. Let’s dive in on what that means.
Planning Increment Planning
We first contemplate sprint packages during our quarterly planning increment (PI) planning session. The primary purpose of these sessions is to revisit and adjust the overall direction of each of our products (what the Scaled Agile Framework calls “value streams”). We market four distinct products that are at different points in their lifecycles. At PI Planning, our executive stakeholders have the opportunity to direct more or fewer teams toward one or another product– say, to accelerate feature development for the newest product that sales is pushing hard, while reducing enhancements on one or more older ones.
We move work, not people, so when one product needs more attention we package that work in the sprints of more than one scrum team. The more intense the push on that product, the more team capacity we allocate, even to the point of putting other products on the shelf.
Even a shelved product has users, though, so as we package new functionality into sprints during PI planning we still reserve some team bandwidth to address emergent client issues on them.
By the end of our two-half days of PI planning, we should have the stories ready for the first sprint in the planning increment. They should be decomposed, refined, conditions of acceptance agreed upon, and estimates made by the team that will do the work. When we have multiple teams working on the same product we often do joint estimation sessions, getting an estimate that all of the teams accept.
We should also have a good idea of what stories will be in the second sprint, some with estimates. And we have an idea of the remaining work in the overall feature-level goals for the planning increment. The teams are asked for a vote of confidence on completing the features in the planning increment. More on that later.
What Could Possibly Go Wrong?
With the roadmap laid out during PI planning, why don’t things go like clockwork for at least the first two sprints? Because priorities change. And agile practices were specifically devised to accommodate these changes
Sales might announce with great pleasure that a new client is about to sign a lucrative contract as long as a specific feature is developed within the next four months. Assuming the feature they’re after is in keeping with the overall direction of the product, no product manager could deny this opportunity to boost sales. There goes the plan!
Client Services or Support reports that a couple clients have started escalating their frustration with a bug that you thought wasn’t that critical. You’re at risk of losing client satisfaction if you don’t fit that bug fix into the next sprint.
Midway through the first sprint of the planning increment, the architects realize they missed something. In order for the new functionality to be scalable and meet performance requirements you’ve got to package some architecture work to harden the backbone. The sweetest features are useless if the system doesn’t perform as expected for the end users.
Marketing comes back from a conference with amazing insights. If we add just enough of these new technologies to the product so that we can say we’re doing it, we’ll get ahead of the competition. This kind of insight might not disrupt the sprint packages in the current planning increment, but you might need to do research spikes to prepare stories for the next PI planning session, and that will steal bandwidth from your current plan.
Factors in Creating a Sprint Package
The product owner uses several data points as well as input from stakeholders to pick which stories to package. The first concern is team velocity: how much work can this team typically complete? Velocity is the average number of story points that they’ve finished over the previous few sprints, omitting outliers.
No single item in a sprint should be more than 50 percent of team velocity. So if a team’s velocity is 27, the largest story they should undertake is one with an estimate of 13 points. They should never be given two 13 pointers. That’s the equivalent of putting all their eggs in one basket. Their package should be one 13 pointer and two or three smaller items that give them room to negotiate if they discover a bad estimate. The smaller the user story, the more accurate the team’s estimate. It would be better to decompose stories so that you never even have any that are half the team’s velocity, but sometimes that’s very difficult while retaining adequate business value to justify the effort.
Items must be independent so that some team members can work on one while others work on another and they won’t step on each other’s code. When multiple teams work on the same product this is even more critical. This can be especially tricky if you address the 50 percent of velocity problem by decomposing a larger story into two or three smaller ones, then package them in the same sprint for the same team. Did you really make them independent? Or did you just divide up the list of acceptance criteria into three and call them different stories? Lazy requirements management has been the downfall of many a sprint.
Team and individual skills and knowledge is the third packaging factor. If one team has a database guru and the other does not, push the data-heavy story to that team. If you’ve only got one team and nobody is great with user experience, you may have to bring in some expertise to work with the team, which will reduce their velocity as they learn from the expert.
Hold about ten percent of bandwidth in reserve. Note that your velocity will already account for this, so if you see that velocity is 30, you don’t reserve ten percent of that and only package 27 points – the 30 point average was accomplished with ten percent of bandwidth already reserved.
But why not consume every possible hour the team has available? Isn’t that wasteful? The effectiveness of Lean practices is widely recognized, and reserving some bandwidth for the unexpected is a Lean principle. In order for a Lean scrum implementation to work, you must trust your team. They aren’t a bunch of lay-abouts trying to avoid work. They’re talented engineers who want to produce a quality product. If they really aren’t, then you’ve got the wrong people, but that’s a different blog post.
During each sprint the team may discover the ability to fix legacy bugs that weren’t packaged. They’re working on a certain part of the code and they can improve it, but it will take more time than they originally planned. That’s an extremely good use of that ten percent you reserved.
Emergencies happen: team members are pulled onto emergent client issues; an additional item “must get done” in this sprint (a bad practice, but sometimes unavoidable); the team miss-estimated so something is much larger than planned; multiple team members get the flu.
“Bench” is not a dirty word in Lean practices, and the time will not go to waste. Even if none of the things listed above happen, team members will use the time productively – on additional testing, on research for upcoming work, on more general learning, on cross-training in areas of the code they don’t yet know. And if they find time for a couple rounds of ping pong that has value too.
Priority Within the Sprint
In that simple model I mentioned at the top, the stack ranked prioritization of the backlog is key, but when you work with sprint packages, the product owner is less focused on the individual priority of each item in the package. They want it all done. However, the team can use individual item priority to help them plan their sprint.
Since the goal of an agile process is to deliver business value, the priority of the items in a sprint should be based on their business value. However, bugs don’t have business value (you already got credit for the value of the feature even though it had the bug, so you don’t get more credit for fixing it). And sometimes fixing a bug is the most important thing to do in a sprint.
Streamline uses Microsoft Team Foundation Server to manage all scrum work, and the form for bugs (rightly) does not include a business value field. So we use the Backlog Priority field, available on both bugs and stories, to prioritize the items in each sprint.
The product owner specifies two or three key deliverables for each sprint that represent its “essence.” These goals can relate to just some, or to all of the items in the sprint package. If there are items in the package that don’t fulfill the goals, they are, by definition, lower priority and the product owner’s stack ranking of the items should reflect that. Successfully meeting the sprint goals, as judged by the product owner, means the sprint is a success, even if the team doesn’t complete all of the packaged stories and bugs. This flies in the face of our metrics, though, which are based on pure data as I’ll discuss in a moment.
At the end of PI planning the teams are asked for a “Fist to Five” confidence vote, fist being “no confidence” and five fingers being “confident.” If the teams vote lower than fours and fives, the product owner leads a discussion of the concerns, and it might lead to some further decomposition of the planned features or a reduction in planned scope.
Managing the Package During the Sprint
After a confident kickoff the team plans the sprint by breaking down the stories and bug into one and two-day duration tasks. When this is done, if the total estimate, in hours, for the tasks is less than the team’s available time, they’re golden. But what if their task estimates add up to more time than they expect to have?
First they review the items in the package. Have they misunderstood anything? Probably not. Most likely as they planned they identified more complexity than they saw during backlog grooming and estimation. It happens, and that’s why there are so many checkpoints in the process.
First the team should re-estimate the problem story – maybe it was estimated at 8 points, but now they think it’s a 13. The team goes back to the product owner to negotiate. They can reduce the scope of one or more items by removing some conditions of acceptance, then re-estimate them to see if they’re back within velocity. Or they can remove one or more items from the sprint. What gets removed is the product owners’ decision. Whether it’s enough is the team’s.
After negotiating, the team does another commitment vote and, hopefully, digs in on the work.
What happens if a team doesn’t finish everything?
The incomplete items are returned to the backlog. They may or may not go into the next sprint – that’s up to the product owner. If something is partially completed, the team is responsible for making the code whole – they might have to shelve changes that are incomplete or untested so that the completed code can be delivered. The team must re-estimate any items that they’ve done some work on, so that their story point estimate reflects the remaining work. Sometimes the estimate won’t change, meaning that the item was underestimated at the start and the team didn’t catch it early on. That’s a sign of an immature team, and something to be addressed in the retrospective.
Traditional project management methodologies run on lots of metrics by collecting lots of data – timesheets, bug tracking, test execution, and so on. At Streamline, the first principal of any metrics around scrum is that we don’t go deeper than the team. We do not look at metrics on individual team members. Obviously we collect data – timesheets, metadata in Team Foundation Server, and so forth. So we are capable of analyzing an individual’s contribution should the need arise, but we do not do that as a matter of course.
Team and product metrics are what matter. Three useful metrics that we look at are:
- Sprint Success – did each team finish what they committed to? This is a data-based metric, not a sprint goals assessment, so it’s a little harsher than the more popular goals-based success. We track whether each team delivered to their commitment, over delivered (by taking on additional items after kickoff), or under delivered. While an under delivery is a failure, we do show how much they did complete rather than simply call it a zero. Sprint success is reported to the organization in a monthly round-up of metrics from all areas.
- Velocity – how much did they get done? As discussed earlier, this is used by the product owner and team to determine how much to package next time. It’s data based, but adjusted to omit outliers (like that sprint when three team members got the flu and they didn’t deliver two stories). Velocity is not formally reported to the organization, but rather used by the product owner in packaging.
- Business Value — Delivering value to our clients is the primary goal of agile. Business value added by product is a useful trailing indicator that you are indeed enhancing the product that’s the highest priority. For that to work, you must have parity across all products and a baseline for each one (“product X was worth 10,000 points of business value and we added another 300 this sprint”). We’re not quite there yet with this data, but it’s in our plan.
How Are You Doing?
How are you doing at managing sprint packages through their lifecycle? Do you track any other metrics that you find valuable? Let us know in the comments.