How to Validate Benefits
Nothing validates benefits like sales, one may assert. Not everything we build, however, is market-facing. Even market-facing products may have back-end projects. Besides, sales is also a function of marketing effectiveness, market conditions, and competitors, so it can be misleading to rely exclusively on it. But perhaps the biggest drawback of having to rely on sales feedback is that it is often too late in the day. Iterative product development calls for benefits validation prior to the product/enhancement being generally available for sale.
On the other hand, usage analytics provide a better and generic way of measuring the increase or decrease in uptake after each release. Each release has to be as small as possible in order to get close to the conditions required for a controlled experiment. Frequent, small releases are only possible with Continuous Delivery and so it becomes a prerequisite for institutionalizing the habit of benefits validation.
Analytics are needed to validate benefits whether the product is internal or market-facing. They need to be factored into development efforts just like how performance and other operational requirements are taken into consideration. Most enterprise IT is lacking in this regard and it may take a while to institutionalize analytics. In the meantime, user surveys may serve as a stop-gap technique of benefits validation.
Avoid Single-Stage Funding
Benefits validation is meant to inform subsequent rounds of funding. If benefits aren't accruing as expected, it may be better to cut losses and divert funds to other projects. However, business cases are sometimes funded at one go. Even if execution is planned as a number of releases, the funds for all releases (meant to realize a business case) is sanctioned at one go. Thus, single-stage funding fails to take advantage of benefits validation.
Single-stage funding is sometimes a side-effect of a tedious approval process. Business cases are commonly written to secure 100% funding to avoid cycling through the approval process multiple times. Why are approval processes so tedious? Perhaps because of big project failures in the past or because of the conviction that a strong approval process would increase the chance of success. It has the makings of a vicious circle.
Portfolio Management in Product-Centric IT
The discussion so far has focused on how benefits validation can drive portfolio management in traditional enterprise IT. It called for some changes:
- Bring product and delivery people to work closer together so as to deliver benefits rather than just deliver scope.
- Institutionalize the use of production analytics to obtain data for benefits validation.
- Adopt Continuous Delivery to be able to make frequent, small releases that allow for granular benefits validation.
- Avoid single-stage funding.
Sign up for CIO Asia eNewsletters.