Article Image

IPFS News Link • Robots and Artificial Intelligence

Facebook's AI Can Caption Photos for the Blind on Its Own

• Wired

MATT KING IS blind, so he can't see the photo. And though it was posted to his Facebook feed with a rather lengthy caption, that's no help. Thanks to text-to-speech software, his laptop reads the caption aloud, but it's in German. And King doesn't understand German.

But then he runs an artificial intelligence tool under development at Facebook, and after analyzing the photo, the tool goes a long way towards describing it. The scene is outdoors, the AI says. It includes grass and trees and clouds. It's near some water. King can't completely imagine the photo—a shot of a friend with a bicycle during a ride through European countryside—but he has a decent idea of what it looks like.

"My dream is that it would also tell me that it includes Christoph with his bike," King says. "But from my perspective as a blind user, going from essentially zero percent satisfaction from a photo to somewhere in the neighborhood of half … is a huge jump."

The 49-year-old King is part of the Facebook Accessibility Team. This means he works to hone the world's most popular social network so that it can properly serve people with disabilities, including people who are deaf, people without full use of their hands, and, yes, people who are blind, like King himself. Though that AI tool is merely a prototype, Facebook plans to eventually share it with the world at large. And that's no small thing. About 50,000 people actively use the social network through Apple Voiceover, a popular text-to-speech system, and the overall population of blind Facebookers is undoubtedly much larger.


Home Grown Food