"We collect information about transactions, about senders, about receivers, and enhance that data with transactional history," he says. "We have a lot of moving parts, and I have to enable real-time information around transactions, credit limits and other policies such as international anti-money-laundering regulations." Thompson explains that in October 2012, he started putting together a system "that could respond to the needs of the business in an economical way."
It went live in May 2013. Thompson uses Informatica and Tibco software to deliver the data sources into a single Hadoop repository and Tableau for visualization. Cloud service provider Cloudera hosts the system. Western Union started out with a 64-node cluster but upgraded to a 128-node cluster nine months after going live because "we kept finding more and more business uses and bringing in live transactional data," he says.
If the company had employed a data warehouse for the same effort, Thompson says, "we would have needed massive amounts of storage, as well as analysts and technicians to code SQL against the database." But because it's folly to run live queries against production systems, the traditional method would have been to take snapshots of data every few days and run queries against those snapshots. "But it would have been expensive to keep replicating the data," he says. "With Hadoop, we have one large instance of all our data in real time, and we can run instantaneous queries without impacting the production system."
Western Union benefits in multiple ways. "We get better consumer protection, because we're ensuring that transactions are evaluated for risk. We now have a better view into adjusting fraud-prevention models, so we have fewer losses," Thompson says. "Best of all, I'm finally delivering real-time information into the hands of people who need it."
Faster disaster response
An insurance claims center after a disaster is like an isolated restaurant when a tour bus rolls in -- a setting that was calm a moment earlier turns chaotic quickly. And these days, the chaos is compounded by the fact that customers can contact their insurers through so many different channels.
"During a typical month, we process approximately 700 claims," says Werner E. Kruck, chief operating officer at Ormond Beach, Fla.-based Security First Insurance. "In the aftermath of a hurricane, that can swell to tens of thousands within days."
In recent years, as the number of customers contacting agents or claims representatives via social media started to increase, Kruck says he grew concerned that his company "might have a difficult time [responding to] an influx of social media messages received outside of our traditional business systems" following a natural disaster. His concern was confirmed after tornados struck Tuscaloosa, Ala., and Joplin, Mo., in early 2011 and again when Hurricane Sandy devastated the Eastern Seaboard in October 2012.
Sign up for CIO Asia eNewsletters.