NSF Convergence Accelerator Track H: Next Generation Augmentative and Alternative Communication Technology Powered by Artificial Intelligence

沟通是人类的一项基本需求，是一项基本人权，是一种基本的人类力量。通过沟通，每个人都能参与并影响日常生活中的变化。对于数以百万计的美国人来说，交流受到言语和语言的限制。这些限制包括缺乏语言，大多数交流伙伴不理解的语言，以及有限的语言理解。有沟通障碍的个人在参与生活，包括教育、职业和社会活动方面受到严重限制。辅助和替代交流（AAC）设备旨在帮助有语言和语言障碍的人表达他们的想法、需求和想法。尽管有证据表明AAC设备可以改善与家人、朋友和交流伙伴的社交互动；参与教育环境；而参与职业生活，现有的技术还没有充分发挥其潜力。挑战包括教用户与AAC设备进行交互，个性化设备以适应每个用户的通信需求，以及使通信尽可能轻松。我们假设，通过利用人工智能（AI）的最新进展，我们可以增强AAC技术，并重新定义用户与之交互的方式，最终对许多有通信限制的人及其通信伙伴（包括他们的家人）的生活产生变革性影响。该融合项目旨在通过融合和多学科的努力，引领新一代人工智能驱动的AAC技术，涉及AAC、人工智能、人机交互和软件设计方面的专家，以及AAC最终用户、护理人员和利益相关者。本项目研究的AAC实施例将是用于移动触摸屏设备的AAC应用程序的原型。设想的人工智能驱动的AAC应用程序将能够(1)获取和分析有关用户及其环境的多模态上下文信息，以提供个性化；(2)根据AAC和包容性设计的最佳实践，并由人工智能驱动，生成动态网格和视觉场景显示，以最大限度地提高通信效率；(3)包括一个能够指导AAC用户的虚拟对话伙伴。(4)支持旨在鼓励频繁使用AAC和促进语言发展的虚拟活动。通过协作、迭代的过程，研究团队将收集利益相关者的意见，对人工智能驱动的AAC概念进行使用启发式探索，并通过快速原型和用户测试来评估这些概念。改善沟通支持将导致更好的教育成果，并增加有沟通限制的个人参与劳动力，从而导致更丰富的社会。该项目旨在为数百万通信受限的美国人提供一个更加公平的世界，他们受益于AAC。项目团队将与技术转移专家和AAC行业领导者进行互动，以制定开发和部署下一代AAC设备的最佳路径，并将其交付给最终用户。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Communication is a basic human need, a fundamental human right, and an elemental human power. It is through communication that all individuals participate and affect change within their daily lives. For millions of Americans, communication is restricted by limitations in speech and language. These restrictions can include a lack of speech, speech that most communication partners do not understand, and limited language understanding. Individuals with communication limitations are severely restricted in their participation in life, including in educational, vocational, and social activities. Augmentative and alternative communication (AAC) devices aim to help people with speech and language limitations express their thoughts, needs, and ideas. Despite the evidence that AAC devices can improve social interaction with family, friends, and communication partners; participation in educational contexts; and participation in vocational life, the existing technology has yet to reach its full potential. The challenges include teaching users to interact with AAC devices, personalizing the devices to fit the communication needs of each user, and making communication as effortless as possible. We hypothesize that by leveraging recent advances in Artificial Intelligence (AI), we can enhance AAC technology and redefine how the users interact with it, ultimately achieving a transformative impact on the lives of many people with communication limitations and their communication partners, including their families. This convergence project aims to spearhead a new generation of AI-powered AAC technology through a convergent and multidisciplinary effort involving experts in AAC, AI, human-computer interaction, and software design, as well as AAC end users, caregivers, and stakeholders. The AAC embodiment studied in this project will be a prototype of an AAC application for mobile touchscreen devices. The envisioned AI-powered AAC application will be able to (1) ingest and analyze multi-modal contextual information about the user and their environment to provide personalization, (2) produce dynamic grids and visual scene displays informed by best practices in AAC and inclusive design and driven by AI to maximize communication efficiency, (3) include a virtual conversational partner able to guide the AAC user, and (4) support virtual activities aimed at encouraging frequent AAC use and promoting language development. Through a collaborative, iterative process, the research team will gather stakeholder input, conduct a use-inspired exploration of AI-powered AAC concepts, and evaluate the concepts through rapid prototyping and user testing. Improved communication support would result in better educational outcomes and increased participation in the workforce for individuals with communication limitations, thus leading to a more abundant society. This project stands to support a more equitable world for the millions of Americans with communication limitations who benefit from AAC. The project team will interact with technology transfer experts and AAC industry leaders to lay out the best path towards developing and deploying the next-generation AAC devices and putting them into the hands of end users.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.