CCRI: ENS: Collaborative Research: Developing the Dialog Ecosystem to Support and Enhance Research in Spoken Dialog Systems

人们在日常生活中都在与对话系统对话。Siri、Alexa等已经成为家喻户晓的名字。但正如这些系统的任何用户所知道的那样，它们远非完美。它们目前所能做的事情种类也很有限。使用数据对于使用机器学习和人工智能技术改进这些系统至关重要，但创建对话系统的公司通常将这些数据留给自己，这使得研究人员更难正确地创建、改进和评估这些系统。创建DialPort的目的是从世界各地的对话系统的真实用户那里收集数据。研究人员可以将他们的系统连接到Portal，或者请求其他人收集的数据。除此之外，对于需要帮助创建对话系统以运行研究和收集数据的研究人员，DialPort DialogEcosystem为他们提供了创建系统的工具和如何使用它们的教程。对于已经拥有系统并希望使用人工计算（通常称为众包）进行测试的研究人员，DialPort DialogEcosystem提供了简单的任务创建和与主要众包网站的连接。为了降低进入该领域的门槛，DialPort对话生态系统通过REAL Challenge帮助培训年轻学生，学生可以在其中想象理想的对话系统并学习如何创建它们。这个项目的成果将最终影响到日常生活中使用对话系统的每一个人。DialPort项目以前由计算机与信息科学与工程（CISE）研究基础设施计划资助，它使口语对话社区能够访问工具、数据和用户。想要创建一个新的对话系统的研究人员可以参考DialPort的网站来获取他们需要的工具。当对话系统启动并运行时，他们将其连接到Portal，以便真正的用户与他们的系统进行通信。DialPort对话生态系统将以多种方式满足不断增长的社区不断变化的需求。更多的工具和教程将提供。真正的用户数据已经开始流向连接到门户的系统，并且为了产生最先进的系统所需的大量数据，这种流量将大大增加。许多研究人员用众包工人测试他们系统的早期版本，但他们不熟悉如何设置任务和运行质量控制，他们需要帮助。DialPort DialogEcosystem将通过其DialCrowd工具响应这些不断变化的需求。创建对话系统的研究人员的领域正在扩大。问答领域现在正在使用聊天机器人作为测试其检索能力的一种手段。在机器学习中，研究人员正在研究自然语言生成，并发现对话系统是可以测试他们工作的工具。对话生态系统将接触到每一个从事对话系统工作的人，并提供一个完整的框架，可以满足他们从创建到评估的需求。除了现有的研究人员，DialPort对话生态系统还将接触到年轻的学生，向他们传授我们的领域。DialPort对话生态系统将：创建手持版本的门户，以解决真实用户和工作人员如何访问门户；大大扩展DialTools，包括包装和教程流行的系统创建工具，扩展DialCrowd，以帮助研究人员创建，评估和分析结果从众工任务，创建一个新的真正的挑战，以帮助高中和本科学生熟悉我们的领域。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

People are talking to dialog systems in everyday life. Siri, Alexa and others have become household names. But as any user of these systems knows, they are far from perfect. They are also currently limited in terms of the types of things they can do. Usage data is essential to improve these systems using machine learning and artificial intelligence techniques, but companies that create dialogue systems often keep this data to themselves, making it harder for researchers to create, improve, and evaluate such systems properly. DialPort was created with the goal of gathering data from real users for dialog systems around the world. Researchers can connect their systems to the Portal or request the data that others have collected. Beyond this, for researchers who need help creating a dialog system that they could use to run studies and collect data, the DialPort DialogEcosystem provides them with access to tools for the creation of their systems and tutorials on how to use them. For researchers who already have systems and want to test them using human computation (often called crowdsourcing), the DialPort DialogEcosystem provides easy task creation and connection to major crowdsourcing sites. And to lower the barrier to entry to the field, the DialPort DialogEcosystem helps train young students with its REAL Challenge in which students can imagine ideal dialog systems and learn how to create them. The results from this project will ultimately impact every person who uses dialog systems in daily life.The DialPort project, previously funded by the Computer and Information Science and Engineering (CISE) Research Infrastructure Program, has given the Spoken Dialog Community access to tools, data and users. Researchers who want to create a new dialog system consult DialPort's website for access to the tools they need. When a dialog system is up and running, they connect it to the Portal to get real users to communicate with their systems. The DialPort DialogEcosystem will keep up with the evolving needs of a growing community in several ways. More tools and tutorials will be available. Real user data has started to flow to the systems connected to the Portal and that flow will increase substantially in order to produce the large amounts of data that are needed by state-of-the-art systems. Many researchers test early versions of their systems with crowdworkers, but they are not familiar with how to set up tasks and run quality control and they need help. DialPort DialogEcosystem will respond to these evolving needs with its DialCrowd tools. The field of researchers who are creating dialog systems is expanding. The field of Question Answering is now using chatbots as a means of testing their retrieval capabilities. In Machine Learning, researchers are working on natural language generation and finding that dialog systems are vehicles that can test their work. The DialogEcosystem will reach out to everyone who works on dialog systems and offer a complete framework that can serve their needs from creation to assessment. Beyond present researchers, the DialPort DialogEcosystem will reach out to young students to teach them about our field. The DialPort DialogEcosystem will: create a handheld version of the Portal to address how real users and workers access the Portal; greatly extend DialTools to include wrappers and tutorials for popular system creation tools, extend DialCrowd to help researchers create, assess and analyze results from Crowdwork tasks, create a new REAL Challenge to help high school and undergraduate students become familiar with our field.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.