Google Photos enables you to find just about anything in your photo collection, including pictures of things that are "delicious." Credit: Mike Elgan
Both Facebook and Google have been working hard at using computers and algorithms to identify people in photos. They've gotten really good at it.
We still don't know what they'll do with that technology. To a large degree, it's up to us. But first, we have to understand what's possible.
Why Facebook doesn't need your face
Facebook is one of the leading organizations in the world developing facial-recognition algorithms. Facebook software can now identify people in photographs as well as people can. Facebook's DeepFace (no, I'm not kidding -- it's called DeepFace) can tell whether the subjects in two different photographs are the same person with 97 percent accuracy. That's even better than the FBI's own Next Generation Identification system.
DeepFace achieves this amazing feat by analyzing faces, turning them into 3D models, then making it possible to recognize the faces from angles and under lighting conditions that are different from those in other photos of the same person. The technology uses more than 120 million parameters, and a page on Facebook's research website explains that the company "trained it on the largest facial dataset to-date, an identity labeled dataset of four million facial images belonging to more than 4,000 identities."
But that's not enough for Facebook. It wants to be able to identify people even when their faces aren't showing. Toward that end, Facebook researchers are developing a system that looks at hairstyle, body shape, posture, clothing and so on.
Facebook can now recognize people whose faces aren't showing with 83 percent accuracy.
Tellingly, the company tried to avoid freaking people out with this research by developing the algorithm using Flickr pics, not Facebook photos.
While Facebook's ability to recognize people is astonishing, so is Google's.
How Google identifies you without identifying you
Everybody oohed and ahhed at Google I/O last month when Google demonstrated the search feature in its newly announced Google Photos offering.
In fact, we Google+ users have been enjoying this capability for years. Google Photo's search engine can not only tell the difference between cats and dogs, but also identify dog breeds and perform other search feats that seem impossible.
It will even find photos based on adjectives that could be used to describe the images they depict. For example, when I searched my own photo collection using the word delicious, it showed me hundreds of pictures I've taken of foods and beverages that were, in fact, delicious. (It also showed a picture I took during a safari in the Masai Mara in Kenya of a cheetah devouring a gazelle -- I guess that was delicious to the cheetah and Google software somehow knew that.)
Sign up for CIO Asia eNewsletters.