CAREER: Technology Assisted Conversations

面对面的交谈是人们相互交流的重要方式，但不幸的是，有数百万人患有妨碍正常交谈的疾病。 该项目将为面临说话挑战的人探索新的实时通信解决方案，包括身体或认知残疾的人，例如通过利用从一个人的对话伙伴获得的隐式和显式上下文输入。 目标是开发技术，改进目前可用的增强和替代通信（AAC）设备，以帮助人们更快，更流畅地说话。 该项目将扩大研究资源到会话交互系统，交付成果包括概率文本输入工具包，AAC用户界面和增强现实会话助手。 项目成果将包括灵活、强大和数据驱动的方法，这些方法可扩展到新的使用场景。 为了扩大其更广泛的影响，该项目将通过外联活动和工作将创建的在线社区，向公众宣传审计咨询委员会。 PI将召集本科生团队开发该项目的软件，他将主持一个关于短信背后技术的暑期青年项目，为女性、残疾学生和来自代表性不足群体的学生提供奖学金。 资助的第一年的研究机会将进一步帮助留住本科生，特别是女性，在computing.This项目将探索对话交互系统的设计空间，通过调查两个系统，以改善使用AAC设备的非发言个人的沟通和系统，提高谁面对其他会话相关的挑战发言个人的沟通。 用途：使用以下各项的上下文敏感预测算法将被考虑：1）关于会话伙伴的回合的语音识别; 2）如由说话者标识确定的伙伴的身份; 3）对话状态信息;以及4）伙伴在移动终端上提出的建议。 用户研究将调查基于合作伙伴的预测的有效性和用户接受度。 将建立新的方法来评估对上下文敏感的审计咨询委员会接口。 将比较训练AAC语言模型对来自现有语料库、模拟AAC用户和实际AAC用户的数据的影响。 这项研究将扩大我们对如何在增强现实中利用会话背景的知识，并将策划一个由AAC用户贡献的公共测试集。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Face-to-face conversation is an important way in which people communicate with each other, but unfortunately there are millions who suffer from disorders that impede normal conversation. This project will explore new real-time communication solutions for people who face speaking challenges, including those with physical or cognitive disabilities, for example by exploiting implicit and explicit contextual input obtained from a person's conversation partner. The goal is to develop technology that improves upon the Augmentative and Alternative Communication (AAC) devices currently available to help people speak faster and more fluidly. The project will expand the resources for research into conversational interactive systems, the deliverables to include a probabilistic text entry toolkit, AAC user interfaces, and an augmented reality conversation assistant. Project outcomes will include flexible, robust, and data-driven methods that extend to new use scenarios. To enhance its broader impact, the project will educate the public about AAC via outreach events and by the online community the work will create. The PI will assemble teams of undergraduates to develop the project's software, and he will host a summer youth program on the technology behind text messaging, offering scholarships for women, students with disabilities, and students from underrepresented groups. Funded first-year research opportunities will further help retain undergraduates, particularly women, in computing.This project will explore the design space of conversational interactive systems, by investigating both systems that improve communication for non-speaking individuals who use AAC devices and systems that enhance communication for speaking individuals who face other conversation-related challenges. Context-sensitive prediction algorithms that use: 1) speech recognition on the conversation partner's turns; 2) the identity of the partner as determined by speaker identification; 3) dialogue state information; and 4) suggestions made by a partner on a mobile device will be considered. User studies will investigate the effectiveness and user acceptance of partner-based predictions. New methodologies will be created for evaluating context-sensitive AAC interfaces. The impact of training AAC language models on data from existing corpora, from simulated AAC users, and from actual AAC users will be compared. This research will expand our knowledge about how to leverage conversational context in augmented reality, and it will curate a public test set contributed by AAC users.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.