Facebook Lets Blind Users ‘See’ Photos With Automatic Alternative Text Feature

Photos are hugely important to Facebook’s appeal and is one of the main reasons that over 1.5 billion people around the globe use the social network so regularly. Today, Facebook is broadening that appeal to help users who are blind or visually impaired to “see” photos on the site by using machine learning to automatically identify objects in the photos and describe them using sound.

The feature, Call Automatic Alternative Text, will begin rolling out in English on Facebook’s iOS app from Tuesday with more languages coming online in the future. The feature will work in conjunction with a screen reader, which is the primary way blind people access the internet. Until now, visually impaired users could only hear who was in the picture if they had been tagged. Now users will hear a description of who and what is in the picture as well as information like what type of food a person might be eating and even if they are smiling.

Facebook says the technology is still in its early stages but it can already reliably identify objects in a range of categories, including transportation (“car,” “boat,” “airplane”), nature (“snow,” “ocean,” “sunset”), sports (“basketball court”) and food (“sushi”). It will also describe people (“baby,” “smiling,” "beard”), and even identify if the picture is a selfie.

An example of what a user might hear with the new system is: “This image may contain: Three people, smiling, outdoors.” The feature will initially only be available in the U.S., U.K., Canada, Australia and New Zealand

“While this technology is still in its early stages, tapping its current capabilities to describe photos is a huge step toward providing our visually impaired community the same benefits and enjoyment that everyone else gets from photos,” the company’s accessibility team said in a post on the site.

While Twitter is seeking to get people to add descriptions to their photos when uploading them, considering that over 2 billion photos are added across Facebook’s range of services on a daily basis (including Facebook, Instagram, WhatsApp and Messenger) making adding a description to every photo would quickly alienate users.

Instead Facebook is leveraging the huge investment it has made in artificial intelligence in recent years and using machine learning to help automatically identify objects within photos.