A slightly easier job is understanding spoken language in limited contexts, such as, for example, banking and telecom call centers.
"We start with a generic set of rules that we know work for, say, the telecommunications industry, and then use that in conjunction with their specific domain," says Chris Ezekiel, CEO of Creative Virtual, a company that processes spoken and written speech for companies like Verizon, Virgin Media, Renault, and the UK's National Rail.
"'Hannah,' for instance, for [UK's] M&S Bank, knows all about their credit cards, loans, and other financial service products," he says.
For companies that deploy virtual assistants like Hannah, the goal is to answer questions that normally are handled by human staff. According to Ezekiel, these virtual agents typically average 20% to 30% success rates, and the systems are continuously updated to learn from previous encounters so that they can handle more queries.
One Creative Virtual client, Telefónica UK, found that their intelligent agent Lucy reduced customer service calls by 10% to 15%. That doesn't mean that she only understands 10% to 15% of questions, says Telefónica knowledge base manager Richard Hagerty. "One of the key questions customers ask is, 'How do I contact customer service?'"
In other cases, Lucy might not yet know the answer, and the company will need to create one. "Maybe we wouldn't answer the question, anyway," he says.
What the company has learned over the past 12 months is that it's better to have one clear answer than to respond with several possible answers. In addition, Lucy needs to become a bit less human, he adds. For example, Lucy can handle a wide variety of personal questions. She says she likes Italian food, for example, has seen Titanic several times, and enjoys tennis and salsa dancing.
"There's a back story that allows a customer to ask personal questions," Hagerty explains. "She lives in Wimbledon, and is engaged to her boyfriend. But some customers believe they are having a chat with a human being. So we are looking at reducing some of the elements of personalization so that our customers' expectations are managed correctly. We want to make it clear to our customers that it's an automated service they're using, not a human being."
Gestures a tough nut to crack
Interface designers looking to translate spoken -- or written -- words into practical goals have a solid advantage over those designing interfaces for gestures or other non-traditional input methods.
That's because designers are already familiar with the use of spoken language. And if they aren't, there is a great deal of research out there about how people use language to communicate, says MIT Media Lab's Holzman. The language of human gestures is much less understood and less studied.
Sign up for CIO Asia eNewsletters.