Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Is big data too big to handle?

Sheila Lam, Computerworld HK | Aug. 7, 2013
Despite the enthusiasm for big data, how many big data conversations have turned into actual implementation?

Among these four factors, the top three challenges relate to technology immaturity and data management. Although these are daunting challenges, they are mainly technology-related concerns, which are usually easier to resolve.

The good news is that the more complicated issues like developing a business case (24.4%) or culture and internal conflicts (14.6%) are ranked much lower. This finding suggests users across the organizations are generally aware the benefit of big data and less effort is required to educate and convince users for investment.

Volume versus variety
In addition to studying motivations and challenges, the survey also drilled down into each of the three Vs to understand the requirements and expectation involved in managing data volume, variety and velocity.

The size of data considered as big data in Hong Kong enterprise appears to match global standards. About two-thirds of local IT professionals expect data volumes to be at least 10TB or more, with close to 20% of them stating that data volumes over 100TB are considered big data.

According to Wikipedia, big data size in 2012 ranged from a few dozen terabytes to many petabytes of data in a single data set. "Big data sizes are a constantly moving target," says Wikipedia. "The target moves due to constant improvement in traditional DBMS technology as well as new databases like NoSQL and their ability to handle larger amounts of data."

As technologies advance, data volume for enterprises will increase. Besides data volume, the survey also tried to quantify the nature of data contributing to this growth. The survey asked respondents two separate questions regarding the data volume generated by different types of data, as well as the types of data expected to undergo big data analysis. 


The general picture indicates that data that are currently generating the larger volume are also expected to be used for big data analysis. 62.7% and 53% of IT readers stated structured transactional data and email, respectively, are generating the largest amount of data. They are also the two most popular types of data to be used for big data analysis.

Rising star: social media content
With most IT users rating the ability to process larger data volumes as the top benefit, data types with larger volume are more likely to be used for advanced analytics.

However, there are exceptions, and social media content is one. Relatively few organizations (26.9%) stated social media is currently generating a large amount of data, but social media is highly rated (40.5%) for big data analysis.

Although the volume of data generated from social media is massive, most organizations have yet to strategically capture this data set for analysis. With the rise of mobility and increasing power of social media, more organizations are expected to turn towards this platform for insight, and big data technologies are expected to play a critical role in the process. The same motives drive higher interest (37.3%) in leveraging big data to analyze digital rich media, like videos, audio, and graphics.


Previous Page  1  2  3  Next Page 

Sign up for CIO Asia eNewsletters.