New Zealand (Commonwealth Union) – Researchers from the University of Canterbury’s HIT Lab NZ in Canterbury are delving into the potential applications of virtual reality (VR) as a valuable tool in remote medical assessments.
In a recent study led by PhD student Dilshani Kumarapeli, alongside Postdoctoral Fellow Dr. Sungchul Jung, Research Associate Dr. Yuanjie Wu, and Professor Rob Lindeman, Director of the Human Interface Technology (HIT) Lab at Te Whare Wānanga o Waitaha | University of Canterbury (UC), the team explored the utility of VR headsets in medical consultations. This innovation becomes particularly valuable in situations where access challenges or risk factors make it impractical for both the doctor and the patient to be physically present in the same room.
The researchers introduced a concept they refer to as an “asymmetric system,” a unique feature that distinguishes their approach from the more conventional uses of VR technology, such as in the realm of video games. Unlike symmetric systems where both users engage with the same system and receive identical information, their asymmetric system tailors the VR experience to each user’s specific requirements.
The research team envisions that this highly personalized system has the potential to strengthen the connection between doctors and patients, even when they are physically separated. By delivering a sensory experience tailored to address each patient’s individual needs, this technology could be particularly beneficial for patients in remote locations, individuals with highly contagious illnesses, or those with mental health conditions that may pose a risk of violent behavior towards others.
In essence, this groundbreaking research demonstrates the exciting possibilities of using VR technology to bridge gaps in healthcare, ultimately providing more effective and personalized medical assessments and care, regardless of geographical or physical constraints.
“The patient needs to feel trust in their clinician, so for them, the system is focused on relaying eye-contact and facial expressions that make them feel connected to the doctor despite being in a different physical location,” explained Professor Lindeman. “For the clinician, it can provide detailed physiological information that the patient might struggle to convey verbally during a remote assessment.”
The headset conveys a wealth of vital data to the doctor, encompassing eye-tracking, facial expressions, as well as pulse and breathing rate information. This data is seamlessly processed by an interconnected software program, empowering the clinician to make more precise and informed diagnoses.
Although the research team has already conducted trials with a group of University of Canterbury students, both Professor Lindeman and Kumarapeli acknowledge the vast potential for employing similar “asymmetric” VR technology in a wide array of scenarios, including educational contexts. However, they intend to take the system a step further by advancing its development for practical implementation in real-world situations.
For Kumarapeli, the most gratifying aspect of the research journey was the creation of the emotion recognition software. Reflecting on this accomplishment, she added that due to the participants wearing the headset, they were unable to capture their entire facial expressions with a camera, so they had to find an alternative solution to address this challenge. He said, “We used a facial capture device and trained a neural network to identify various key emotions. These emotions were then relayed to the clinician, to help in managing the diagnosis session. I really enjoyed the journey of developing the neural network and applying the trained modules to this real-life scenario.”
The scientists got their backing from the Science for Technological Innovation National Science Challenge and the work is under preparation for publication. Dr Jung who was the key investigator on the study has since then taken up a continuing Assistant Professor position in the USA. The potential of VR technology has been taken up by researchers over the years. As the technology advances, more and more benefits are likely to emerge in the years ahead.
