Students at Singularity University have created an app that allows blind people to identify the world using machine vision technology. The app, Aipoly, is an intelligent assistant for the visually impaired that empowers them to explore and understand their surroundings through computer vision and audiofeedback.
The user takes a picture that is automatically uploaded to the Aipoly servers, where it is analysed and tagged, and a description is sent back and read out loud using texttospeech. This means that blind people may be able to see what their kids are wearing each day, recognise street signs, find objects that are out of reach, and have freedom to purchase gifts for their friends by themselves.
“The power is in helping us construct the mental picture. And not everybody has the same skill at creating mental images,” says Steve Mahan, president of the Santa Clara Blind Centre and Google’s selfdriving car’s first user. “Most of us are trying to do [that]. Knowing where we are is sometimes more than an address.”
The machine vision algorithm is optimised for use by the visually impaired with training in street signs and objects commonly used by blind people. Machine vision, or computer vision, is an exponential technology that has more than doubled in accuracy between 2012 and 2013. Convolutional neural networks are used to identify the elements within a picture and neural image caption generation to feed back a semantic description of its content.
There are 285M visually impaired people in the world and in the next 5 years, two thirds of them will become smartphone users. As for the bigger vision, “developing this technology further could help us identify and search for objects around our homes and outdoors like we do with websites online,” says Aipoly cofounder Alberto Rizzoli.
Singularity University was founded in 2008 by Ray Kurzweil, a pioneer in blind technology having created optical character recognition (OCR) and the first texttospeech synthesizer, with over 40 years of experience in the field.
“This complements the work that Ray Kurzweil has done,” says Aipoly cofounder and 2012 Young Australian of the Year Marita Cheng. “In every focus group, people mention a Kurzweil technology they use to get about their daily lives.”
Singularity University students learn about using exponential technologies to impact the lives of a billion people within 10 years. The Aipoly technology will be showcased at Singularity University’s Demo Day on 18 August at NASA AMES Research Park in Mountain View.
Aipoly is now looking for beta testers from around the world. Beta testers of all visual abilities (including fully sighted and blind) are sought.