A few weeks ago VMware released a blast at Amazon's recently released total cost of ownership (TCO) bake-off model that compared AWS cloud services with the on-premise vSphere suite of virtualization products. VMware called the AWS online tool "biased and inaccurate."
Oh my! That's sort of like Manet criticizing Seurat because his dots are too small.
The truth is, as a rule vendor-based TCO, ROI, and Value models are biased and inaccurate. Every major and many minor IT providers have focused on TCO as a key function in their marketing and sales strategies. They often become the basis for ROI projections and business case justifications. And virtually all of their models are biased and inaccurate "out of the box."
I have reviewed dozens of vendor models over the years. I have also used the VMware model extensively in the field and as a developer. My team and I have also dug into the AWS model. Biased and inaccurate, every one of them.
So where do the biases and inaccuracies lie? (No pun intended.)
All of these models are based on spreadsheets, some of these are also modified to work online. Errors can be in the logic, the arithmetic, the output, the complexity or the assumptions.
The assumptions are a likely spot for inaccuracies and one that VMware targets in it criticism of Amazon. There is a litany of allegations ranging from inaccurate server numbers to useful life of the installed base to having any installed base at all. These assumptions come from "industry data", "field experience" and "expert knowledge." Very few of these assumptions are documented. Proceed with caution when accepting these numbers as valid for your environment.
Many of these models are very complex with multiple tabs, many links between tabs, scalars, hidden tables and factors that can easily be broken during development or in field use. The more complex, the greater potential for arithmetic or logic errors that might be difficult to isolate.
So what does an IT decision maker to do?
Here are five absolutely necessary steps for buyers that will make a vendor-based TCO model and your business justification case to senior management more defensible:
1. Know your own numbers and take ownership of them. Vendor assumptions are really just placeholders for customer specific costs. Every one of them should be replaced with your numbers if possible. I gained some notoriety at Gartner for two things. First I created the first IT TCO model for PCs. Then, on stage in front of 100s of clients, I stated that the Gartner numbers were B.S. — that we just made them up! The lesson here was that any cost or time estimates are biased and inaccurate unless you own them.
Sign up for CIO Asia eNewsletters.