So, how do you make agile measureable in a consistent and defensible way? Actually, describing how to get hard measurements isn't that hard. Here are several metrics you'll want to consider and the questions you should ask in order to get your answers.
Sprint variances from stated schedule:
- Did feature development stop on time?
- Was integration done every night?
- Did the final tests start on time?
- Did the sprint complete on schedule?
Sprint compliance with coding standards:
- What percent of new variables has data definitions?
- What percent of the classes or modules have unit tests?
- What percent of lines of code were covered by tests?
- How many test records did their tests use?
- How many outcomes, or variations on expected results, were evaluated by the tests?
Sprint variances for stated budget (or level of effort):
- Did the sprint consume unbudgeted resources such as staff developers, contractors or test resources?
- Did it not consume budgeted resources that could or should have been allocated elsewhere?
- Did the sprint break any deployment or security rules?
- Did the release go through the required audits and reviews?
- Did the release generate any items that need to be remediated in future sprints?
Sprint business value:
- How many "value points" were supposed to have been delivered by the sprint, and how many actually were? (Note that value points are purposely subjective. That way, the business unit can prioritize functionality by whatever has the most impact to the way they run their operation.)
- How much of the sprint effort was infrastructure and refactoring to keep the code flexible to accommodate business needs in an upcoming sprint?
- Can the team continue to do sprints in this way without team burnout or other side effects of "heroic effort"?
- Has the team created a brief postmortem document and lessons learned on personnel, organizational and technical issues?
Soft Measurement: Get Users (Including Executives) Involved
One key success factor in agile is user engagement throughout the sprint. The only way you get real quality, usability and user adoption is by having users involved.
Measuring the number of hours during the sprint that users were actively involved is a good start. Here's the best part, though: It can't just be worker-bees from the business unit.
Why does an executive need to be actively engaged in agile? Of course, sponsorship and championing are important to motivating the organization, pulling them into the change management that is inevitable with significant IT projects.
But there's more we need from the execs, too. They need to articulate the priorities and rationales at a level of detail below platitudes and bullet points. The sprint teams need execs to help make the tough priority calls, providing the users with enough guidance and decision-making authority so that the worker bees can make decisions without being over-ridden.
Sign up for CIO Asia eNewsletters.