Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Microsoft creates mood-sensing tech for the HoloLens augmented-reality headset

Mark Hachman | May 4, 2015
Microsoft is still innovating with the HoloLens, while also showing off practical applications like design and surgery.


Microsoft is still innovating with the HoloLens, while also showing off practical applications like design and surgery.

Microsoft has patented technology that allows wearers of augmented-reality glasses like its HoloLens to interpret the emotional reactions of the people they're viewing.

The patent - granted to Microsoft the day before its Build conference kicks off where the company's message about the HoloLens is more about practical applications than new developments (see below) - covers a "see-through, head-mounted display and sensing devices" that "detect audible and visual behaviors of a subject in a field of view of the device."

That, of course, sounds like the HoloLens, an augmented-reality headset Microsoft has shown reporters behind closed doors, and probably will bring to market, eventually. Virtual-reality devices like the Oculus Rift fit over the user's eyes, enclosing their field of view. But augmented-reality devices like Google Glass and the HoloLens superimpose a head-up display over the real-world objects that a user sees. One of the differences between Glass and the HoloLens is that the latter projects its images in color, and at a higher resolution than Glass.

It appears Microsoft believes the hardware must not only "see" but understand what it's seeing. Because people interact with each other so much, mood interpretation seems like a powerful tool. But will people see the tech as too intrusive and creepy, thus hampering widespread social use? My guess is yes.

How it works
The version of the HoloLens that Microsoft has shown also includes a depth camera, so it can "scan" objects in front of the viewer. The same depth camera is part of the Microsoft Kinect, which can interpret data like a user's heart rate and skin temperature by closely observing his or her face. Microsoft's patent suggests scenarios in which the headset itself performs the interpretation, but also works in conjunction with a smartphone or other device.

Microsoft's patent offers a variation that interprets facial gestures (smiles, frowns, narrowed eyes) as indicators of a particular emotional state. Although the real-world examples Microsoft uses to illustrate the patent include a speaker gauging the emotional response of his audience, another stands out as somewhat creepy: "Other social situations in which the technology may be useful include romantic situations involving a one-on-one relationship between individuals."

Creep-factor aside, the technology could help those with Asperger's, autism, or other social disorders by interpreting emotional and social clues. Or, conceivably, the mood sensing could be used during sales calls or negotiations as a tool to see how far a client could be pushed before backing out. Finally, one might imagine surreptitiously scanning opponents at a poker table to figure out whether they're bluffing.


1  2  Next Page 

Sign up for CIO Asia eNewsletters.