CHS: Medium: Transforming Scientific Presentations with Co-Presenter Agents

虽然期刊和会议文章被认为是最正式和最持久的科学交流形式，但口头陈述对科学至关重要，因为研究人员、从业人员、媒体和公众通过口头陈述了解最新发现，从而参与和受到启发，并在其中获得科学声誉。然而，尽管计算机和通信媒体的技术进步了几十年，口头科学报告的基础自20世纪80年代微软的PowerPoint等软件问世以来并没有进步。PI在这个项目中的目标是通过使用一个智能的、自主的、真人大小的、动画的联合主持人代理，与人类主持人合作，在现场观众面前准备和发表他或她的演讲，从而彻底改变媒体辅助的口头演讲，特别是STEM演讲。PI的试点研究表明，受众接受这一概念，并且该技术对非英语为母语的个人（可能占美国人口的21%）特别有效。项目成果将首先在高等教育中进行部署和评估，既可以作为提供STEM讲座的教学工具，也可以作为科学学生学习如何进行更有效口头陈述的培训工具（这可能会激励后代从事科学事业）。本研究将基于人类-代理协作理论，其中使用实时语音和手势识别监控人类演示者，也监控观众反馈，并且代理，演示媒体和人类演示者（通过智能可穿戴提词器提示）都是动态编排的，以最大限度地提高观众参与度，沟通和说服力。该项目将为实时人机协作和通信模型做出基础的理论贡献。它将探索人类和智能体如何协同工作，通过语音、手势和各种演示媒体与不同的观众进行有效沟通，从而增强原本只能“单飞”的科学家演说家的能力。这项工作将推进人工智能和计算语言学，通过扩展对话系统，以涵盖共同主持人和观众之间的混合主动、多方对话。它将通过推动手势、韵律和近体语的动态生成来促进有效的公共演讲和轮流，从而影响虚拟代理的最新技术。它还将为人机交互领域做出贡献，通过开发人类主持人与自主联合主持人代理及其演示媒体互动的新方法，包括使用可穿戴用户界面有效提示人类主持人的方法。

Although journal and conference articles are recognized as the most formal and enduring forms of scientific communication, oral presentations are central to science because they are the means by which researchers, practitioners, the media, and the public hear about the latest findings thereby becoming engaged and inspired, and where scientific reputations are made. Yet despite decades of technological advances in computing and communication media, the fundamentals of oral scientific presentations have not advanced since software such as Microsoft's PowerPoint was introduced in the 1980's. The PI's goal in this project is to revolutionize media-assisted oral presentations in general, and STEM presentations in particular, through the use of an intelligent, autonomous, life-sized, animated co-presenter agent that collaborates with a human presenter in preparing and delivering his or her talk in front of a live audience. The PI's pilot studies have demonstrated that audiences are receptive to this concept, and that the technology is especially effective for individuals who are non-native speakers of English (which may be up to 21% of the population of the United States). Project outcomes will be initially deployed and evaluated in higher education, both as a teaching tool for delivering STEM lectures and as a training tool for students in the sciences to learn how to give more effective oral presentations (which may inspire future generations to engage in careers in the sciences).This research will be based on a theory of human-agent collaboration, in which the human presenter is monitored using real-time speech and gesture recognition, audience feedback is also monitored, and the agent, presentation media, and human presenter (cued via an intelligent wearable teleprompter) are all dynamically choreographed to maximize audience engagement, communication, and persuasion. The project will make fundamental, theoretical contributions to models of real-time human-agent collaboration and communication. It will explore how humans and agents can work together to communicate effectively with a heterogeneous audience using speech, gesture, and a variety of presentation media, amplifying the abilities of scientist-orators who would otherwise be "flying solo." The work will advance both artificial intelligence and computational linguistics, by extending dialogue systems to encompass mixed-initiative, multi-party conversations among co-presenters and their audience. It will impact the state of the art in virtual agents, by advancing the dynamic generation of hand gestures, prosody, and proxemics for effective public speaking and turn-taking. And it will also contribute to the field of human-computer interaction, by developing new methods for human presenters to interact with autonomous co-presenter agents and their presentation media, including approaches to cueing human presenters effectively using wearable user interfaces.