CHS: Medium: Transforming Scientific Presentations with Co-Presenter Agents

虽然期刊和会议文章被认为是最正式和持久的科学交流形式，但口头报告对科学至关重要，因为它们是研究人员，从业者，媒体和公众了解最新发现的手段，从而获得参与和启发，并在科学声誉中取得。 然而，尽管计算机和通信媒体的技术进步了几十年，但自20世纪80年代微软的PowerPoint等软件问世以来，口头科学报告的基础并没有进步。 PI在这个项目中的目标是通过使用智能，自主，真人大小，动画的共同主持人代理，与人类主持人合作，在现场观众面前准备和提供他或她的演讲，从而彻底改变媒体辅助的口头演讲，特别是STEM演讲。 PI的试点研究表明，受众接受这一概念，该技术对非英语母语的人（可能占美国人口的21%）特别有效。 项目成果将首先在高等教育中部署和评估，既作为提供STEM讲座的教学工具，也作为理科学生学习如何进行更有效口头陈述的培训工具（这可能会激励后代从事科学事业）。这项研究将基于人类与智能体合作的理论，其中，使用实时语音和手势识别来监视人类演示者，还监视观众反馈，并且代理、演示媒体和人类演示者（经由智能可穿戴电传打字机提示）都被动态编排以最大化观众参与、交流和说服。 该项目将为实时人机协作和通信模型做出基础性的理论贡献。 它将探索人类和代理人如何共同努力，使用语音，手势和各种演示媒体与异质观众进行有效沟通，增强科学家演说家的能力，否则他们将“单飞”。“这项工作将通过扩展对话系统来促进人工智能和计算语言学，以包括共同主持人及其观众之间的混合主动性，多方对话。 它将影响虚拟代理的最新技术水平，通过推进手势，韵律和有效的公开演讲和话轮转换的语法的动态生成。 它还将有助于人机交互领域，通过开发新的方法，人类主持人与自主的共同主持人代理及其演示媒体进行交互，包括使用可穿戴用户界面有效提示人类主持人的方法。

Although journal and conference articles are recognized as the most formal and enduring forms of scientific communication, oral presentations are central to science because they are the means by which researchers, practitioners, the media, and the public hear about the latest findings thereby becoming engaged and inspired, and where scientific reputations are made. Yet despite decades of technological advances in computing and communication media, the fundamentals of oral scientific presentations have not advanced since software such as Microsoft's PowerPoint was introduced in the 1980's. The PI's goal in this project is to revolutionize media-assisted oral presentations in general, and STEM presentations in particular, through the use of an intelligent, autonomous, life-sized, animated co-presenter agent that collaborates with a human presenter in preparing and delivering his or her talk in front of a live audience. The PI's pilot studies have demonstrated that audiences are receptive to this concept, and that the technology is especially effective for individuals who are non-native speakers of English (which may be up to 21% of the population of the United States). Project outcomes will be initially deployed and evaluated in higher education, both as a teaching tool for delivering STEM lectures and as a training tool for students in the sciences to learn how to give more effective oral presentations (which may inspire future generations to engage in careers in the sciences).This research will be based on a theory of human-agent collaboration, in which the human presenter is monitored using real-time speech and gesture recognition, audience feedback is also monitored, and the agent, presentation media, and human presenter (cued via an intelligent wearable teleprompter) are all dynamically choreographed to maximize audience engagement, communication, and persuasion. The project will make fundamental, theoretical contributions to models of real-time human-agent collaboration and communication. It will explore how humans and agents can work together to communicate effectively with a heterogeneous audience using speech, gesture, and a variety of presentation media, amplifying the abilities of scientist-orators who would otherwise be "flying solo." The work will advance both artificial intelligence and computational linguistics, by extending dialogue systems to encompass mixed-initiative, multi-party conversations among co-presenters and their audience. It will impact the state of the art in virtual agents, by advancing the dynamic generation of hand gestures, prosody, and proxemics for effective public speaking and turn-taking. And it will also contribute to the field of human-computer interaction, by developing new methods for human presenters to interact with autonomous co-presenter agents and their presentation media, including approaches to cueing human presenters effectively using wearable user interfaces.