AI is transforming sightseeing for visually impaired users

AI is transforming sightseeing for visually impaired users

Breakthrough is changing the way people with visual impairments experience the world, giving them tools to discover, understand and experience the beauty of unknown places like never before.

Study: The AI ​​system makes it easier for people with blindness and low vision to interpret and experience unfamiliar environments. Photo credit: Angel Santana Garcia/Shutterstock.com

A team of researchers from China developed an artificial intelligence (AI) system that may be able to help visually impaired people explore, understand and enjoy unfamiliar environments. The study is published in theNature Portfolio Journal Artificial Intelligence.

background

Exploring natural environments such as parks has a significant positive impact on physical and mental health. However, people with low vision or blindness are often excluded from these benefits because adequate tools are not available to proactively engage with them.

Existing assistive solutions designed to guide visually impaired individuals focus primarily on providing functional support such as navigation and obstacle avoidance, allowing them to passively engage with nature.

Visually impaired people often feel helpless while exploring unfamiliar environments. This usually means they rely on family members, friends or volunteers for support, which affects their ability to actively explore and understand unfamiliar environments, as well as remember and communicate with other visually impaired people about their journey.

A team of researchers in China developed an AI-powered system called Viptour to provide visually impaired individuals with a sense of independence in unfamiliar environments.

How does Viptour work?

Viptour is an AI-driven system that includes a series of lightweight, portable, consumer-grade devices (a camera and a smartphone) and a novel deep learning algorithm network called Focusformer. Efficient multi-sensory interaction techniques such as audio and hierarchical tactile interaction increase the interaction between visually impaired users and the Viptour system.

Focusformer considers aesthetics, freshness (novelty) and basic needs (including navigation and safety) as key factors to extract meaningful information from complex, unfamiliar environments and eliminate redundant visual details. This reduces cognitive load on visually impaired users.

Focusformer transforms large amounts of information into a structured, sparse and hierarchical personalized diagram. Based on this well-structured diagram, Focusformer interacts with visually impaired users through a smartphone application, understands their preferences and provides personalized support through an adapter.

It is self-trained with thousands of public tourism videos from sighted tourists, which is beneficial for effectively reducing aesthetic bias.

The Viptour system also has options for recording, saving and sharing experiences, facilitating emotional communication between visually impaired people and promoting the exchange of knowledge and experiences in their social networks.

Viptours' technical innovation lies in its Multi-T Focus Focusformer network. This approach uses a background subnetwork to filter out frequently seen objects, an attraction subnetwork to identify highlights, a freshness subnetwork to discover novel features, and a subnetwork trained on surveys conducted with visually impaired participants. These subnetworks combine to select, rank, and present the most relevant information to each user.

The Viptour system also uses a Blv-in-the-Loop adapter that updates its recommendations in real-time based on individual user metrics such as “likes” and “dislikes,” enabling personalization.

User opinion on Viptour

The Viptour system was tested on 33 people with blindness or visual impairment, and self-reported emotional experiences were collected for analysis.

In terms of assistive performance, the study found that the Viptour system was effective in helping visually impaired individuals actively explore and thoroughly understand unfamiliar environments, empowering them with accurate and long-lasting memories, and enabling them to communicate with their colleagues.

Through detailed analysis of self-reported experiences, the study found that participants who used Viptour successfully achieved a 67.9% increase in positive emotional response, a 94.7% increase in arousal, a 772.73% increase in cognitive mapping accuracy, and a 200% increase in long-term memory retention.

In user evaluations, the Viptour system's usability scores were consistently above 80 out of 100, comparable to or better than other aids for the visually impaired.

Physiological measures, including electrodermal activity and heart rate variability, showed significant improvements with Viptour use, indicating improved emotional engagement.

Investigate significance

The study highlights the potential uses of the AI-driven Viptour system in providing a pleasant and memorable experience to the visually impaired while actively exploring unfamiliar environments. These experiences can significantly improve their emotional state and improve their overall quality of life.

Existing evidence suggests that presenting organized and engaging information can improve a person's enjoyment and facilitate deeper memory retention. People tend to process well-structured and meaningful information, which makes their experiences more enjoyable and memorable.

This human tendency can be explained by the concept of cognitive fluency, which indicates that clear and organized information presentation reduces cognitive load in individuals. It then helps them channel mental resources to understand and integrate the content. This improved processing fluency leads to a positive response as people perceive the information more pleasantly.

Furthermore, the interaction between novel and familiar information influences the impact of organized and interesting information on memory. Novel information stimulates curiosity and improves attention, while familiar information provides cognitive comfort and coherence.

Presenting the information in a structured and engaging way can balance novelty and familiarity, which helps maintain the individual's interest and engagement.

Focusformer's self-supervised training with thousands of unlabeled public tourism videos effectively captured cognitive language knowledge and revealed the statistical relationships between different concepts in tourism scenes. This approach eliminates potential bias in tour preference labeling and trains the model to extract only relevant contextual information.

These personalized design considerations from Focusformer have enabled the Viptour system to successfully model desired cognitive language skills, thereby improving the tourism experience for visually impaired individuals.

It is worth noting that Viptour's impact depends on the quality of its underlying AI techniques such as object recognition and semantic graph generation. Future improvements to these methods could further improve the performance of the system.

Download your PDF copy now!

Sources: