AR/MR software for improving communication and education outcomes of minimally verbal autistic people

点击翻译按钮获取中文摘要

There are at least 3 million autistic children and adults in North America alone who cannot communicate effectively using speech. Not being able to share their thoughts, beliefs, and desires is arguably the most significant aspect of their disability, severely limiting their ability to participate in educational, social, employment, and community settings. Additionally, the financial impact of autism is immense. Studies suggest that in Canada the total direct medical, non-medical, and loss of productivity costs associated with autism are at least $30 billion annually. Thus, offering minimally verbal autistic individuals effective alternative ways of communication can have transformative socio-economic impacts. Unfortunately, most existing communication interventions have been shown to be ineffective. One limitation of traditional approaches to communication interventions is that they do not account for the significant motor and sensory challenges that many minimally verbal autistic individuals face. For example, producing intelligible speech requires the precise motor coordination of the vocal apparatus. As a result, minimally verbal autistic individuals may be better able to learn how to communicate in other ways, including written form. This project is developing new software solutions based on Augmented/Mixed Reality (AR) technologies to improve communication and education outcomes for this population. The key innovation is the design of an immersive system that systematically improves motor, i.e., pointing, skills of minimally verbal individuals whilst engaging them with challenging, age-appropriate educational material and literacy activities. The system will leverage wearable sensors to monitor the stress levels of individuals and tailor the virtual environment with accommodations to address sensory challenges. The software arising out of this project will enable minimally verbal individuals to communicate fluently by pointing at letters. This would open educational, social, and employment opportunities that most autistic individuals are never afforded. This will in turn lead to better inclusion and considerable economic benefits.