Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

GUEST BLOG: Tackle big data with little bites

Bob Lewis | Aug. 23, 2012
A small team, a virtual server, and an agile approach to data analytics will reap better results than a big team with a hard deadline

The role of the decision maker is to raise questions and to have a clear idea of what sort of analysis can provide useful answers. The role of the statistical analyst is to take this clear idea and turn it into statistical tests of available data. The role of the data analyst is to track down where the needed data lives, extract the data, massage it into a usable form, and load it into the private environment.

Nothing about what they do should have any permanence -- quite the opposite. Right now you want smart people to muck about in the data, getting their hands dirty up to their elbows so that they can get a feel for the process. You also want to create an environment of haves and have-nots: You want the company's other decision makers to become jealous because they aren't getting the same attention.

That's when you bring your team of three up for air and ask whether the kinds of questions being asked and answered have stabilized enough to turn their ad hoc work into a production environment. Assuming it has, that's their next task.

Lather, rinse, repeat
Once you have one project stable and ongoing, ask your setup team what they could have done to get to production more quickly, if they had to do it all over again. Because they're smart people, they'll have ideas.

Then you can assign the data analyst and statistical analyst to work with the next decision maker in line, to go through the process again, while still providing support to the first one -- you don't want to lose what you've built, after all. Go through this a few times and you'll have generated some solid business value while helping the whole company learn its way into a more sophisticated approach to analytics.

With this growing semi-mess of analytics-oriented data marts, you'll also be in a much better position to design the "real" system the company needs -- and to explain to the executive team what this more elegant system will cost and why it's worth investing time, money, and effort to build.

Or you might not -- this constantly churning collection of data marts might turn out to be exactly what the company needs. Just because it ain't pretty doesn't mean it ain't functional, after all.


Previous Page  1  2  3 

Sign up for CIO Asia eNewsletters.