Machine learning technologies can help companies spot suspicious user behaviors, malicious software, and fraudulent purchases -- but even as the defensive technologies are getting better, attackers are finding ways to get around them.
Many defensive systems need to be tuned, or tune themselves, in order to appropriately respond to possible threats.
Smoke alarms that go off each time someone microwaves popcorn get replaced with less sensitive ones, or are moved farther away from the kitchen.
Old-school crowbar-and-ski mask crooks already know this.
"If there's a motion detector and I ride my bike by innocently and set off their alarm, and do that every day for a month, they'll either turn the motion detector off or recalibrate it," said Steve Grobman, Intel Security CTO at Intel. "That gives me the opportunity to break in."
When the same approach is used against machine learning systems, it's called flooding, he said.
The thing to remember is that cyberdefense isn't like, say, predicting the weather.
"If you're using AI to better track hurricanes, as your accuracy evolves, the laws of physics don't suddenly say, 'we're going to change the way water evaporates'," he said. "In cybersecurity, there's a human on the other end who has the objective to make the model fail."
With flooding, attackers increase signals, sometimes gradually, to the point where the attackers can slip in under the cover of the legitimate activity. Or a distributed denial of service attack can tie up resources.
To deal with this, companies need to look beyond just data analysis.
"We're dealing with a changing landscape, and machine learning and AI can only go so far in dealing with these issues and we'll need some human ingenuity," said Zulfikar Ramzan, CTO at RSA Security. "And it's not enough to just have a data science background -- you need an intersection of data science and domain expertise."
For now, at least, it takes human expertise to understand that the smoke detector was too close to the microwave, or that the guy riding the bike past the house every night at 2 a.m. and throwing a rock near the house to set off the alarm is someone to be wary of.
In cybersecurity terms, that requires understanding how the business works, and whether particular changes in behavior make sense or might indicate suspicious behaviors.
Similarly, domain expertise can help defenders spot attempts to manipulate the data sets that are being used to train machine learning systems.
Malware writers might create a large number of legitimate applications that share the characteristics of the malicious software that they play to write. Rogue employees might adjust their behavior so that when they carry out their nefarious actions they don't get picked up as suspicious.
Sign up for CIO Asia eNewsletters.