Predictive analytics modelling relies on algorithms that tend to be far more complex than more traditional statistical systems. They can be difficult to explain.
The retail industry often uses data science to better predict stock replenishment requirements for weekly item orders. The results can amaze, but there are so many factors to take in that the process itself is often hard to communicate.
"It's just impossible to explain to someone who cannot grasp data complexity, but because it cannot be explained, you cannot decide how good it is just based on your common sense or business intelligence," says Zavalishina. "You need to make sure that you know what it is you want to improve, and how you measure results.
"It's not creative. It's specifics and what it tried to predict or optimise. It's like dealing with mathematicians. You ask the question and then you will receive exactly the answer to this question."
If your question is wrong don't expect the right answer. It's a surprisingly common problem, as companies often lack thorough planning on their objectives and the measurement of assessing them.
"We were working with this big retail company and the asked us to build a model which would predict how much of each and every item will sell the next week," Zavalishina recalls. "We tried it with one item, but the problem was they realised that [the prediction] is practically no use for them."
Their model was precise, but the company was ordering its product in packages of six rather than as individual items. If the prediction called for seven items next week, they would need to answer a different question. Should they buy one or two? It may appear a small change, but it meant they had started at the wrong place. The model became entirely different, because the parameters for optimisation had shifted.
Data science requires careful planning. The company received the right answer, but should have asked a different question.
Failure on the route to success
The optimisation model provided to another retailer suggested that the expensive and unusual products they rarely sold weren't worth ordering at all. The decision was mathematically logical, but that doesn't mean it made business sense. Such items can be crucial to the shop's identity and customer base.
"You are pretty much guaranteed that with your first data science project or machine learning project you will need to get back and rethink what the metrics are and what the goals are," says Zavalishina.
Yandex usually recommends customers begin with projects that are very specific and short, to avoid the risk of a long-term investment in a project that could have meaningless results. This method allows companies to make piece-by-piece improvements across the board.
Sign up for CIO Asia eNewsletters.