CHS: Small: Deep Integration of Crowds and AI for Robust, Scalable, and Privacy-Preserving Conversational Assistance

CHS：小型：人群和人工智能的深度集成，提供强大、可扩展且保护隐私的对话协助

• 批准号: 1816012
• 负责人: Jeffrey Bigham
• 金额: $ 50万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816012&HistoricalAwards=false
• 关键词: CHS; Small; Deep; Integration; Crowds

This research will use the recently deployed crowd-powered conversational assistant, Chorus, as a scaffold to develop technical components that allow it to automate itself over time. Chorus introduces a hybrid intelligence model in which humans and machines collaboratively power a single intelligent system. This is unlike both fully-automated approaches which are limited in terms of the domains they cover, and individual human-based conversational support which does not scale. Conversation is interactive communication. When people converse with one another, they build and refine a shared context that makes finding and making sense of information efficient and more effective. Computers capable of engaging users in natural conversations about arbitrary topics would revolutionize how, when, and where people have access to information. Despite many successes, computers are still far from being able to converse naturally across general domains. Systems resulting from this research will be robust enough and scalable enough to be used in real world domains. These types of hybrid systems may lead to new, generally applicable models that are useful in real-time human computation and natural language understanding. This work will inform a better understanding of how automated agents can learn from crowd-powered systems in order to gradually assume more responsibility over time.Creating a robust, general-purpose dialog system from the bottom up is difficult because it requires solving multiple hard problems at once. This project employs a complementary top-down approach that will (1) use the growing Chorus data set to train automatic responders, (2) facilitate integration of existing task-specific dialog systems, (3) develop learning systems to sample among integrated dialog systems and choose the best to respond, (4) develop learning systems to choose the best responses from among automated and human suggestions, (5) develop learning systems able to recommend relevant elements from the user's history based on context, (6) develop crowd-powered systems for allowing users to safely control their devices, and (7) develop crowd-powered systems that allow users to safely access private repositories such as their email. Integral to this work is the interplay between computers and people. Central goals are to better understand how computers and people can complement the work of one another; learn how people can teach computers to be better in the difficult domain of robust dialog, and develop novel approaches for applying human computation when the crowd is handling confidential information or has control of a physical device such as a user's mobile phone. Lessons learned from exploring the top-down approach of introducing a crowd-powered conversational agent that is gradually replaced by automation may apply generally to other hard problems. This approach may allow research topics to be explored before successful computational approaches have been developed for foundational problems, such as learning how to properly curate persistent memory before having the ability to create reliable conversational assistants.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这项研究将使用最近部署的人群驱动的会话助理Chorus作为支架，开发技术组件，使其能够随着时间的推移自动化。Chorus引入了一种混合智能模型，在这种模型中，人类和机器协同为单个智能系统提供动力。这与全自动方法不同，全自动方法在其覆盖的领域方面受到限制，而基于人类的个人会话支持则无法扩展。 对话是一种交互式沟通。当人们相互匡威时，他们建立和完善了一个共享的环境，使寻找和理解信息变得更加有效。能够让用户参与关于任意主题的自然对话的计算机将彻底改变人们获取信息的方式、时间和地点。尽管取得了许多成功，但计算机仍然远远不能在一般领域之间进行自然对话。从这项研究中产生的系统将是足够强大和可扩展的，足以在真实的世界领域使用。这些类型的混合系统可能会导致新的，普遍适用的模型，在实时人类计算和自然语言理解是有用的。这项工作将有助于更好地理解自动代理如何从群体驱动系统中学习，以便随着时间的推移逐渐承担更多的责任。自下而上创建一个强大的通用对话系统是困难的，因为它需要同时解决多个难题。该项目采用了一种互补的自上而下的方法，该方法将（1）使用不断增长的Chorus数据集来训练自动应答器，（2）促进现有任务特定对话系统的集成，（3）开发学习系统，以在集成对话系统中进行采样并选择最佳响应，（4）开发学习系统，以从自动和人工建议中选择最佳响应，（5）开发能够基于上下文从用户的历史中推荐相关元素的学习系统，（6）开发允许用户安全地控制他们的设备的群体动力系统，以及（7）开发允许用户安全地访问诸如他们的电子邮件之类的私人存储库的群体动力系统。这项工作不可或缺的是计算机和人之间的相互作用。 中心目标是更好地了解计算机和人如何能够相互补充工作;了解人们如何能够教计算机在强大对话的困难领域变得更好，并开发新的方法，用于在人群处理机密信息或控制物理设备（如用户的移动的手机）时应用人类计算。从探索自上而下的方法中吸取的经验教训，即引入一个逐渐被自动化取代的群体驱动的会话代理，可以普遍适用于其他困难的问题。这种方法可以让研究课题探索成功的计算方法已经开发出的基础问题，如学习如何正确地策划持久性记忆之前，有能力创建可靠的会话assistant.This奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

项目成果

Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation

• DOI: 10.18653/v1/2021.findings-acl.338
• 发表时间: 2021-06
• 期刊: Journal of Mathematical Analysis and Applications
• 影响因子: 1.3
• 作者: Prakhar Gupta;Yulia Tsvetkov;Jeffrey P. Bigham

InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning

• DOI: 10.18653/v1/2022.emnlp-main.33
• 发表时间: 2022-05
• 作者: Prakhar Gupta;Cathy Jiao;Yi-Ting Yeh;Shikib Mehri;M. Eskénazi;Jeffrey P. Bigham

Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References

• DOI: 10.18653/v1/w19-5944
• 发表时间: 2019-07
• 期刊: ArXiv
• 作者: Prakhar Gupta;Shikib Mehri;Tiancheng Zhao;Amy Pavel;M. Eskénazi;Jeffrey P. Bigham