Microsoft's New iOS App Is Able To Describe The World To Blind People

Microsoft released a new app for iOS devices called Seeing AI. The app uses computer vision to provide verbal description for users who are blind or visually impaired.

The Seeing AI app was developed by Research AI, an incubation hub inside Microsoft Research. Microsoft actually first showed off a prototype of the app back in March of last year during its Build conference.

The Seeing AI app is very simple to use. Once the app is open, users will first have to select which channel they want to use. After that, users will just have to point their iPhone camera in their desired direction.

Point the iPhone’s camera to an object, and the app will be able to describe to the user what it is. Pointing it at a product’s bar code will have the app describing what it is, and even provide additional information about how to use it.

READ: Bridging America's Digital Divide: Why Microsoft Is Spending $10 Billion To Bring Internet To The Masses

Another useful feature here is that the Seeing AI app can also scan/read documents and read signs for users. The app is also capable of recognizing currency bills and user’s friends. The app is able to recognize people as long as they’re part of the users saved friends.

The app is also able to provide instructions to users if the document, sign or object they want described isn’t in-frame. The app will simply tell users which part of an object or document isn’t visible.

Microsoft’s Seeing AI app can also work with other apps too. One example shown off by Microsoft is that it can describe images on Twitter. Users will just have to import the image to the Seeing AI app in order for it to work.

The Seeing AI app is able to do all of these using artificial intelligence. The app also uses neural networks in order for it to identify and recognize objects in the real world. This is the same kind of tech being used by other companies to power self-driving cars and drones, as pointed out by The Verge.

Another great thing about the Seeing AI app is that most of its basic functions are made on the device itself. This means that users won’t always need a stable internet connection in order to take advantage of the app’s most important features.

READ: Tech Support Scam: UK Investigation Leads To Arrests In Phony Microsoft IT Calls

That being said, the Seeing AI app does come with a couple of experimental features which require a connection to Microsoft’s cloud. The first experimental feature is scene descriptions. This will let the Seeing AI app recognize what’s happening in a picture that the user just captured. The second one is handwriting recognition, and it functions exactly as it’s described.

The Seeing AI app for iOS devices is available now from the Apple App Store for free in the United States, Canada, India, Hong Kong, New Zealand and Singapore. Microsoft says that it’s also heading to more countries soon, but didn’t give out an exact timeline. Unfortunately, there’s no word yet on an Android OS version of the app.