NSF Convergence Accelerator Track H: Next Generation Augmentative and Alternative Communication Technology Powered by Artificial Intelligence

沟通是人类的基本需求，是一项基本人权，也是一种基本的人类力量。正是通过沟通，所有人都参与并影响其日常生活中的变化。对于数百万美国人来说，沟通受到言语和语言的限制。这些限制可能包括缺乏言语，大多数沟通伙伴不理解的言语以及有限的语言理解。有沟通障碍的人在参与生活，包括教育、职业和社会活动方面受到严重限制。增强和替代通信（AAC）设备旨在帮助有言语和语言限制的人表达他们的想法，需求和想法。尽管有证据表明AAC设备可以改善与家人、朋友和沟通伙伴的社交互动;参与教育环境;以及参与职业生活，但现有技术尚未充分发挥其潜力。这些挑战包括教用户与AAC设备交互，个性化设备以满足每个用户的通信需求，以及尽可能轻松地进行通信。我们假设，通过利用人工智能（AI）的最新进展，我们可以增强AAC技术并重新定义用户与其交互的方式，最终对许多通信受限的人及其通信伙伴（包括他们的家人）的生活产生变革性的影响。该融合项目旨在通过融合和多学科的努力，引领新一代人工智能驱动的AAC技术，涉及AAC，人工智能，人机交互和软件设计专家，以及AAC最终用户，护理人员和利益相关者。在这个项目中研究的AAC实施例将是用于移动的触摸屏设备的AAC应用的原型。设想的AI驱动的AAC应用程序将能够（1）摄取和分析有关用户及其环境的多模态上下文信息以提供个性化，（2）生成动态网格和视觉场景显示，这些信息由AAC和包容性设计中的最佳实践提供，并由AI驱动，以最大限度地提高通信效率，（3）包括能够指导AAC用户的虚拟会话伙伴，（4）支持旨在鼓励经常使用AAC和促进语言发展的虚拟活动。通过协作、迭代的过程，研究团队将收集利益相关者的意见，对人工智能驱动的AAC概念进行使用启发式探索，并通过快速原型设计和用户测试来评估这些概念。改善通信支持将带来更好的教育成果，并使通信受限的个人更多地加入劳动力队伍，从而使社会更加富足。该项目旨在为数百万受益于AAC的通信受限的美国人提供一个更公平的世界。该项目团队将与技术转让专家和AAC行业领导者互动，为开发和部署下一代AAC设备并将其交付给最终用户制定最佳路径。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Communication is a basic human need, a fundamental human right, and an elemental human power. It is through communication that all individuals participate and affect change within their daily lives. For millions of Americans, communication is restricted by limitations in speech and language. These restrictions can include a lack of speech, speech that most communication partners do not understand, and limited language understanding. Individuals with communication limitations are severely restricted in their participation in life, including in educational, vocational, and social activities. Augmentative and alternative communication (AAC) devices aim to help people with speech and language limitations express their thoughts, needs, and ideas. Despite the evidence that AAC devices can improve social interaction with family, friends, and communication partners; participation in educational contexts; and participation in vocational life, the existing technology has yet to reach its full potential. The challenges include teaching users to interact with AAC devices, personalizing the devices to fit the communication needs of each user, and making communication as effortless as possible. We hypothesize that by leveraging recent advances in Artificial Intelligence (AI), we can enhance AAC technology and redefine how the users interact with it, ultimately achieving a transformative impact on the lives of many people with communication limitations and their communication partners, including their families. This convergence project aims to spearhead a new generation of AI-powered AAC technology through a convergent and multidisciplinary effort involving experts in AAC, AI, human-computer interaction, and software design, as well as AAC end users, caregivers, and stakeholders. The AAC embodiment studied in this project will be a prototype of an AAC application for mobile touchscreen devices. The envisioned AI-powered AAC application will be able to (1) ingest and analyze multi-modal contextual information about the user and their environment to provide personalization, (2) produce dynamic grids and visual scene displays informed by best practices in AAC and inclusive design and driven by AI to maximize communication efficiency, (3) include a virtual conversational partner able to guide the AAC user, and (4) support virtual activities aimed at encouraging frequent AAC use and promoting language development. Through a collaborative, iterative process, the research team will gather stakeholder input, conduct a use-inspired exploration of AI-powered AAC concepts, and evaluate the concepts through rapid prototyping and user testing. Improved communication support would result in better educational outcomes and increased participation in the workforce for individuals with communication limitations, thus leading to a more abundant society. This project stands to support a more equitable world for the millions of Americans with communication limitations who benefit from AAC. The project team will interact with technology transfer experts and AAC industry leaders to lay out the best path towards developing and deploying the next-generation AAC devices and putting them into the hands of end users.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.