Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Where did that data come from? And do you want fries with it?

Ben Rothke | Nov. 5, 2013
Factors like participant compensation may taint results, says Ben Rothke.

Suggestions and Conclusion
Does it make sense that a CIO of a large financial services firm would spend hours completing surveys in order to get Starbucks or iTunes gift card? Such a CIO would likely have such items thrown at them by vendors trying to get on their calendar.

Metadata is data about data. When it comes to surveys, find out the data about the data. How exactly are they obtaining the data? How are they validating that the respondents are qualified to answer the questions?

This is not to say all data is bad or every survey data should be discarded. But you can't take it at face value.

So how do you get strategic answers you can rely and take action on? One solution is to join a group where you know the people you are dealing with rather than simply relying on blind data. Practitioner-based IT research services such as Wisegate (of which I am a member) and IANS offer the ability to directly interact with your peers. Networking at industry conferences is another method.

Of course, my suggestion of the above to groups is what is known as selection bias. And as Pete Lindstrom of Spire Security observed, "in my experience, every concern with numerical surveys has a similar problem in the qualitative environment. The cool thing about numbers is that it is very easy to see the biases and problems. I consider that a plus, not a minus. In qualitative world, it all gets masked, even though there is a very long list of these psychological scenarios."

Getting good answers to hard technology questions is not easy. If you find a better way, let me know.


Previous Page  1  2  3 

Sign up for CIO Asia eNewsletters.