AR/MR software for improving communication and education outcomes of minimally verbal autistic people

仅在北美就有至少300万自闭症儿童和成人不能用语言进行有效交流。不能分享他们的想法、信仰和愿望可以说是他们残疾的最重要方面，严重限制了他们参与教育、社会、就业和社区环境的能力。此外，自闭症的经济影响是巨大的。研究表明，在加拿大，与自闭症相关的直接医疗、非医疗和生产力损失的总成本每年至少为300亿美元。因此，为最低限度的自闭症患者提供有效的替代沟通方式可以产生变革性的社会经济影响。不幸的是，大多数现有的沟通干预措施已被证明是无效的。传统的沟通干预方法的一个局限性是，它们不能解释许多最低限度语言自闭症患者所面临的重大运动和感觉挑战。例如，发出清晰的言语需要发声器官的精确运动协调。因此，语言能力最低的自闭症患者可能能够更好地学习如何以其他方式交流，包括书面形式。该项目正在开发基于增强/混合现实（AR）技术的新软件解决方案，以改善这一人群的沟通和教育成果。关键的创新是一个沉浸式系统的设计，它系统地提高了运动，即指向，语言能力最低的个体的技能，同时吸引他们具有挑战性的，适合年龄的教育材料和识字活动。该系统将利用可穿戴传感器来监测个人的压力水平，并量身定制虚拟环境，以应对感官挑战。由这个项目产生的软件将使最低限度的口头个人能够通过指向字母流利地交流。这将为大多数自闭症患者提供教育、社会和就业机会。这将带来更好的包容性和可观的经济效益。

There are at least 3 million autistic children and adults in North America alone who cannot communicate effectively using speech. Not being able to share their thoughts, beliefs, and desires is arguably the most significant aspect of their disability, severely limiting their ability to participate in educational, social, employment, and community settings. Additionally, the financial impact of autism is immense. Studies suggest that in Canada the total direct medical, non-medical, and loss of productivity costs associated with autism are at least $30 billion annually. Thus, offering minimally verbal autistic individuals effective alternative ways of communication can have transformative socio-economic impacts. Unfortunately, most existing communication interventions have been shown to be ineffective. One limitation of traditional approaches to communication interventions is that they do not account for the significant motor and sensory challenges that many minimally verbal autistic individuals face. For example, producing intelligible speech requires the precise motor coordination of the vocal apparatus. As a result, minimally verbal autistic individuals may be better able to learn how to communicate in other ways, including written form. This project is developing new software solutions based on Augmented/Mixed Reality (AR) technologies to improve communication and education outcomes for this population. The key innovation is the design of an immersive system that systematically improves motor, i.e., pointing, skills of minimally verbal individuals whilst engaging them with challenging, age-appropriate educational material and literacy activities. The system will leverage wearable sensors to monitor the stress levels of individuals and tailor the virtual environment with accommodations to address sensory challenges. The software arising out of this project will enable minimally verbal individuals to communicate fluently by pointing at letters. This would open educational, social, and employment opportunities that most autistic individuals are never afforded. This will in turn lead to better inclusion and considerable economic benefits.