NEW YORK, 24 MARCH 2011 -- As real-time and batch analytics evolve using big data processing engines such as Hadoop, corporations will be able to track our activities, habits and locations with greater precision than we ever thought possible.
"It will change our existing notions of privacy. A surveillance society is not only inevitable, it's worse. It's irresistible," said Jeff Jonas, a distinguished engineer with IBM. Jonas spoke to a packed house of several hundred people Wednesday at the Structure Big Data 2011 conference here.
For businesses, the ability to determine where people are by using geo-locational data will help them personalize advertising and marketing materials disseminated via the Web. For example, if a company knows a customer is in Aruba, it won't bother showing him ads for restaurants in New York; it might market sunblock or scuba-diving excursions instead.
Knowing where people are will also enable companies to accurately determine which potential customer is which. For example, if there are five people in the U.S. who have the same name and the same date of birth but live in different cities, it would be possible to verify the identity of each individual by determining their locations at a given time.
"Just look at the last 10 years of address histories ... it is very telling if this is the same person or not," Jonas said. "Two different things cannot occupy the same space at the same time."
Jonas said 600 billion electronic transactions are created in the U.S. every day, and many of those transactions come from geo-locational data generated by cell phones, which through cellular towers, triangulate a person's exact location at any time. Wireless providers have that data in real time.
By looking at data over a period of years, corporations can know how you spend your time, where you work, and who you typically spend time with.
"This is super food [for big data analytics]," Jonas said. "With 87% certainty, I can tell you where you'll be next Thursday at 5:35 p.m."
"Big data" -- an industry term that refers to large data warehouses -- includes machine- and human-generated data such as computer system log files, financial services electronic transactions, Web search streams, e-mail metadata, search engine queries and social networking activity. In 2010 alone, 1.5 zetabytes of that kind of data was created, most of it machine-generated. Companies filled their data center storage systems with about 16 exabytes of that data last year, according to Jason Hoffman, founder and chief scientist at cloud software provider Joyent.
Sign up for CIO Asia eNewsletters.