CCRI: ENS: Collaborative Research: Developing the Dialog Ecosystem to Support and Enhance Research in Spoken Dialog Systems

人们在日常生活中与对话系统对话。Siri、Alexa和其他公司已经成为家喻户晓的名字。但正如这些系统的任何用户都知道的那样，它们远不是完美的。就他们目前能做的事情类型而言，他们也是有限的。使用数据对于使用机器学习和人工智能技术改进这些系统至关重要，但创建对话系统的公司通常将这些数据保密，这使得研究人员更难正确地创建、改进和评估此类系统。创建DialPort的目的是为世界各地的对话系统收集来自真实用户的数据。研究人员可以将他们的系统连接到门户网站，或者请求其他人收集的数据。除此之外，对于需要帮助创建可用于运行研究和收集数据的对话系统的研究人员，DialPort对话生态系统为他们提供了创建系统的工具和如何使用这些工具的教程。对于已经拥有系统并希望使用人类计算(通常称为众包)进行测试的研究人员来说，DialPort对话生态系统提供了轻松的任务创建和与主要众包网站的连接。为了降低进入该领域的门槛，DialPort对话生态系统帮助培训年轻学生进行真正的挑战，在挑战中，学生可以想象理想的对话系统，并学习如何创建它们。这个项目的结果最终将影响到每一个在日常生活中使用对话系统的人。DialPort项目之前由计算机和信息科学与工程(CEISE)研究基础设施计划资助，使口语对话社区能够访问工具、数据和用户。想要创建一个新的对话系统的研究人员可以咨询DialPort的网站，以获得他们所需的工具。当对话系统启动并运行时，他们将其连接到门户网站，以使真正的用户与其系统进行通信。DialPort对话生态系统将在几个方面跟上不断增长的社区的不断发展的需求。将会有更多的工具和教程可用。实际用户数据已开始流向与门户网站相连的系统，这一流量将大幅增加，以便产生最先进的系统所需的大量数据。许多研究人员与众包工作者一起测试他们的系统的早期版本，但他们不熟悉如何设置任务和运行质量控制，他们需要帮助。DialPort DialogEcosystem将通过其DialCrowd工具响应这些不断变化的需求。正在创建对话系统的研究人员的领域正在扩大。问答领域现在正在使用聊天机器人作为测试它们检索能力的一种手段。在机器学习中，研究人员正在研究自然语言生成，并发现对话系统是测试他们工作的工具。对话生态系统将接触到所有从事对话系统工作的人，并提供一个完整的框架，可以满足他们从创建到评估的需求。除了目前的研究人员，DialPort对话生态系统还将接触到年轻学生，向他们传授我们这个领域的知识。DialPort对话生态系统将：创建门户的手持式版本以解决真实用户和工作人员访问门户的方式；大幅扩展DialTools以包括流行系统创建工具的包装器和教程；扩展DialCrowd以帮助研究人员创建、评估和分析众包工作任务的结果；创建新的真正的挑战以帮助高中生和本科生熟悉我们的领域。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

People are talking to dialog systems in everyday life. Siri, Alexa and others have become household names. But as any user of these systems knows, they are far from perfect. They are also currently limited in terms of the types of things they can do. Usage data is essential to improve these systems using machine learning and artificial intelligence techniques, but companies that create dialogue systems often keep this data to themselves, making it harder for researchers to create, improve, and evaluate such systems properly. DialPort was created with the goal of gathering data from real users for dialog systems around the world. Researchers can connect their systems to the Portal or request the data that others have collected. Beyond this, for researchers who need help creating a dialog system that they could use to run studies and collect data, the DialPort DialogEcosystem provides them with access to tools for the creation of their systems and tutorials on how to use them. For researchers who already have systems and want to test them using human computation (often called crowdsourcing), the DialPort DialogEcosystem provides easy task creation and connection to major crowdsourcing sites. And to lower the barrier to entry to the field, the DialPort DialogEcosystem helps train young students with its REAL Challenge in which students can imagine ideal dialog systems and learn how to create them. The results from this project will ultimately impact every person who uses dialog systems in daily life.The DialPort project, previously funded by the Computer and Information Science and Engineering (CISE) Research Infrastructure Program, has given the Spoken Dialog Community access to tools, data and users. Researchers who want to create a new dialog system consult DialPort's website for access to the tools they need. When a dialog system is up and running, they connect it to the Portal to get real users to communicate with their systems. The DialPort DialogEcosystem will keep up with the evolving needs of a growing community in several ways. More tools and tutorials will be available. Real user data has started to flow to the systems connected to the Portal and that flow will increase substantially in order to produce the large amounts of data that are needed by state-of-the-art systems. Many researchers test early versions of their systems with crowdworkers, but they are not familiar with how to set up tasks and run quality control and they need help. DialPort DialogEcosystem will respond to these evolving needs with its DialCrowd tools. The field of researchers who are creating dialog systems is expanding. The field of Question Answering is now using chatbots as a means of testing their retrieval capabilities. In Machine Learning, researchers are working on natural language generation and finding that dialog systems are vehicles that can test their work. The DialogEcosystem will reach out to everyone who works on dialog systems and offer a complete framework that can serve their needs from creation to assessment. Beyond present researchers, the DialPort DialogEcosystem will reach out to young students to teach them about our field. The DialPort DialogEcosystem will: create a handheld version of the Portal to address how real users and workers access the Portal; greatly extend DialTools to include wrappers and tutorials for popular system creation tools, extend DialCrowd to help researchers create, assess and analyze results from Crowdwork tasks, create a new REAL Challenge to help high school and undergraduate students become familiar with our field.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.