It's a common refrain these days: Data-driven transformation isn't just about technology adoption, it requires changing the very culture of your organization — the way your people think, interact and work. But what does that really mean?
Over the past few years, online marketplace Etsy has made that transformation. Data and metrics now support the entire company's operations, says CTO Kellan Elliott-McCrea. At Etsy, there's a saying: "If it moves, graph it."
"And if it doesn't move, graph it anyway because it may make a run for it," Elliott-McCrea says.
About five years ago, Etsy initiated a relaunch of its site and a complete rearchitecturing of its technology, in large part to improve its search capabilities.
Big but unique
Etsy is a huge online marketplace, but it is also unique in that many of the items for sale on Etsy are one-of-a-kind, something buyers have never seen before. Search relevance and recency is a challenge for all online commerce, but the challenge is orders of magnitude bigger at Etsy. There are 29 million unique items available on Etsy and search accounts for nearly 30 percent of all traffic and an even greater percentage of sales, Elliott-McCrea says.
Connecting the right buyers and the right sellers at the right time is "incredibly difficult," Elliott-McCrea says.
"We got into analytics trying to solve the search relevance problem," he adds. "Now it powers nearly everything we do."
At the time, the decision to undertake such a major change was not easy. Search relevance was an issue, but business was humming.
"It was a working business," Elliott-McCrea says. "It was a great community. It was making money. People loved it. But we weren't able to support change."
The site would experience frequent outages when IT delivered new features and there were silos of data everywhere.
"It wasn't an environment set up to learn, discover and iterate," Elliott-McCrea says. "It wasn't a good deploy story. It was scary to change things."
Setting the IT organization up to "learn, discover and iterate" is exactly what Etsy set out to do. But to learn you've got to try new things. And that means making mistakes.
Elliott-McCrea notes that the traditional ways software developers gain confidence in their solutions are quality assurance and unit testing.
"Both of those are approaches for saying the product we built does the thing we think it's going to do and it doesn't break," he says.
But unit tests generally operate in a local environment, while Etsy is built on distributed systems. And traditional QA tends to slow down the development process.
"We wanted developers feeling that they owned the success or failure of their process," Elliott-McCrea says. "In a distributed system, one of the ways to gain confidence is metrics. I'm going to launch this feature to just one percent of users, just admins, just in the U.S. Did the graph move? That's what drove the metrics-driven approach — the ability to gain confidence in a distributed system with lots of iteration."
Sign up for CIO Asia eNewsletters.