Encouraged by bats use of echolocation, scientists have established smart glasses that convert visual data into unique sound representations that increase the capability of blind and vision-impaired individuals to understand their surroundings. The technology may well be world-changing for the visually impaired.
Assistive technology includes manipulative technologies to allow individuals with sensory incapacities to overcome obstacles in their everyday lives. Blindness or low vision (BLV), mainly affects a person’s capability to execute activities of daily living and participate in social activities and communications.
A comprehensive area of assistive technology research is connected with using visual, haptic/tactile and auditory response as a means of enhancing the senses. Now, scientists at the University of Technology Sydney (UTS) have established next-gen smart glasses that interpret visual data into distinct sound icons, so-called “acoustic touch”, as a way of assisting BLV people to ‘see’.
Chin-Teng Lin, one of the study’s co-authors informs that smart glasses typically use computer vision and supplementary sensory information to interpret the wearer’s surrounding into computer-synthesized language. “However, acoustic touch technology locates objects, producing unique sound representations as they enter the device’s area of view. For example, the sound of rustling leaves may suggest a plant, or a buzzing sound might signify a mobile phone.”
Encouraged by the way bats use echolocation, producing a sound wave that bounces off an object, returning an echo that delivers information about the object’s size and distance, the scientists set about developing their smart glasses, which they’ve termed Foveated Audio Device (FAD).
The FAD includes a set of amplified reality glasses and an OPPO Find X3 Pro Android phone. The Unity Game Engine 2022 control the audio input and camera/head tracking output of the glasses. Together, this allows the FAD to convert objects into distinct sound representations when they entered the device’s field of view.
The scientists tried their glasses on 14 adult applicants, seven who were BLV and seven blindfolded sighted participants who represented as a control. The study comprised of a training stage, a seated task where the FAD was utilized to scan and sonify objects on a table, and a standing task that indicates the performance of the FAD when participants were mobile and looking for an item in a disorderly environment. Four objects were utilized in the research: a bowl, book, cup and bottle.
It was found that the wearable device greatly improved the capability of BLV individuals to identify and reach for items without too much mental struggle.
The auditory response empowers operators to recognize and reach for items with amazing accuracy, informed by the lead and corresponding author of the study Howe Yuan Zhu. The results indicate that acoustic touch has the potential to present an active and wearable method of sensory augmentation for the visually impaired community.
With some alteration, the acoustic touch technology could become an essential part of assistive technologies, permitting BLV individuals to understand their environment better than before.
