When it comes to big data projects, the biggest challenge most organizations face, regardless of size, is staffing, says Peter Guerra, a principal in Booz Allen Hamilton's Strategic Innovation Group.
Organizations struggle to find trained data scientists, or to find the money to retrain staff, Guerra says. To address that need, Booz Allen Hamilton released Explore Data Science last week at O'Reilly Strata Conference + Hadoop World in New York City.
"We have seen the rate of adoption of big data really start to skyrocket in the past couple of years," Guerra says. "It's gone from clients who are wondering whether they should pay attention to this or not, to 'How do I move forward? What's the right team? What's the right set of technology?'"
Build Core Team to Address Big Data Challenges
Many CIOs either have big data pilots going today or will shortly, Guerra says. "As they move to production deployments, we have seen some gaps," he adds. "A lot of CIOs get inundated with messages from vendors: 'If you buy my solution, everything will be fine.' But there's no magic bullet. We see more and more CIOs who are willing to piece together a set of technologies so they can grow and expand over time."
Guerra says building out a big data infrastructure from what has become a bewildering array of open source technologies and tools, as well as solutions from proprietary vendors, is no easy task. Despite representing a major consulting firm, Guerra strongly recommends that organizations establish an internal team with expertise in the big data technologies.
"We have a lot of great people that you can buy by the hour, and we'll come in and give you our expertise, but what we always encourage our clients to ... establish their own team," he says. "Hire it in and focus on having a core set of engineers. What we can then do is come in and share our lessons learned. At the end of the day, the best thing for a lot of our clients is to have that core team that they can then augment with consultants as needed."
That's just the beginning, Guerra says once you have the infrastructure in place, you need people who can take that data and transform it into insight. "We haven't seen a lot of training focused on getting the data out. How do you extract insight, not just BI?"
From Statistics to Data Science, With the Help of Gamification
Guerra says the browser-based Explore Data Science training program provides anyone who's taken a high school statistics class with hands-on experience in data science techniques such as the following:
- Distance metrics
- Dimensionality reduction
- Genetic algorithms
Sign up for CIO Asia eNewsletters.