AR/MR software for improving communication and education outcomes of minimally verbal autistic people

仅在北美，就有至少300万患有自闭症的儿童和成年人无法有效地使用语言进行交流。不能分享他们的想法、信仰和愿望可以说是他们残疾的最重要的方面，严重限制了他们参与教育、社会、就业和社区环境的能力。此外，自闭症对经济的影响是巨大的。研究表明，在加拿大，与自闭症相关的直接医疗、非医疗和生产力损失的总成本每年至少为300亿美元。因此，为言语最少的自闭症患者提供有效的替代沟通方式可能会产生变革性的社会经济影响。不幸的是，大多数现有的沟通干预措施都被证明是无效的。传统的沟通干预方法的一个局限性是，它们没有考虑到许多言语能力较弱的自闭症患者面临的重大运动和感觉挑战。例如，要想发出清晰易懂的语音，就需要发声设备精确的动作协调。因此，言语最少的自闭症患者可能能够更好地学习如何通过其他方式进行沟通，包括书面形式。该项目正在开发基于增强/混合现实(AR)技术的新软件解决方案，以改善这一群体的沟通和教育成果。关键的创新是设计了一种身临其境的系统，系统地提高了说话最少的人的运动技能，即指向技能，同时让他们参与具有挑战性的、适合年龄的教育材料和识字活动。该系统将利用可穿戴传感器来监控个人的压力水平，并为虚拟环境量身定做住宿设施，以应对感觉挑战。由该项目产生的软件将使言语最少的人能够通过指向字母来流畅地交流。这将打开大多数自闭症患者从未获得的教育、社会和就业机会。这反过来将带来更好的包容性和可观的经济效益。

There are at least 3 million autistic children and adults in North America alone who cannot communicate effectively using speech. Not being able to share their thoughts, beliefs, and desires is arguably the most significant aspect of their disability, severely limiting their ability to participate in educational, social, employment, and community settings. Additionally, the financial impact of autism is immense. Studies suggest that in Canada the total direct medical, non-medical, and loss of productivity costs associated with autism are at least $30 billion annually. Thus, offering minimally verbal autistic individuals effective alternative ways of communication can have transformative socio-economic impacts. Unfortunately, most existing communication interventions have been shown to be ineffective. One limitation of traditional approaches to communication interventions is that they do not account for the significant motor and sensory challenges that many minimally verbal autistic individuals face. For example, producing intelligible speech requires the precise motor coordination of the vocal apparatus. As a result, minimally verbal autistic individuals may be better able to learn how to communicate in other ways, including written form. This project is developing new software solutions based on Augmented/Mixed Reality (AR) technologies to improve communication and education outcomes for this population. The key innovation is the design of an immersive system that systematically improves motor, i.e., pointing, skills of minimally verbal individuals whilst engaging them with challenging, age-appropriate educational material and literacy activities. The system will leverage wearable sensors to monitor the stress levels of individuals and tailor the virtual environment with accommodations to address sensory challenges. The software arising out of this project will enable minimally verbal individuals to communicate fluently by pointing at letters. This would open educational, social, and employment opportunities that most autistic individuals are never afforded. This will in turn lead to better inclusion and considerable economic benefits.