Collaborative Research: Participatory Sensemaking with Embodied Co-Creative Agents

该奖项支持开发计算架构（称为PACE）的研究，以模拟人类与具身智能机器之间的具身和共同创造行为。人类在许多环境中创造性地合作（共同创造），例如，与同伴跳舞，与孩子玩假装游戏，或在工程设计会议上进行头脑风暴。尽管这种经历很常见，但它很少定义我们与智能机器的互动。这个项目探索了即兴、合作和共同创作的舞蹈，作为共同创作伙伴之间非接触的身体互动形式。该项目基于创造性意义理论，该理论将创造性实践视为一个动态的社会过程，在这个过程中，个体在不同的认知状态之间交替，因为他们在新环境中理解并回应他人的行为。该项目将通过研究高级舞者之间的人类共同创造实践来促进科学进步，以更好地了解人类的共同创造行为，并为共同创造AI技术的设计和开发提供信息。项目团队将利用这种理解来开发可以与人类共同创造的人工智能代理。项目成果将包括能够复制人类共同创造方面的新型机器算法，以及人类舞者与体现的机器智能进行共同创造舞蹈的公共表演。这项工作将为更有效的接口的长期发展提供信息，这些接口将在广泛的应用中涉及到集成的思维、机器和运动功能，例如物理治疗、设计头脑风暴或未来与家庭机器人的体验。该项目还通过外展和指导以及公共人工智能表演支持K-12、本科、研究生和公共教育。该项目的目标是开发一个模块化的、可重用的系统，用于构建具体的协同创造人工智能。项目团队将把现代舞作为一个应用领域，因为它的从业者经过正式培训，通过具体化的非接触身体互动来探索、表达和合作。项目团队将首先对即兴表演中人类舞者的视频和手势数据进行定性分析。这个分析将集中在理解舞者如何通过与彼此和环境的互动（即通过一个称为参与式意义制造的过程）来建立理解。然后，该团队将在一个架构中正式确定他们的发现，以创建虚拟的、具体化的代理，这些代理可以在即兴创作中感知、学习和产生运动。为了演示和完善这一架构，该团队将开发一个共同创造的人工智能，在当代舞蹈中接近专家级别的参与式意义，并训练这个代理创建一个精心策划的即兴合作伙伴。代理人将在排练和演出中接受评估。这个项目的主要贡献将是a)第一个开放获取的带注释的二元运动数据集；B)更好地理解人类舞者如何共同创作；c)舞者与AI进行训练和即兴表演的界面；d)基于共同创造力的实证研究，在运动相关领域创建共同创造性人工智能的架构。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award supports research to develop a computational architecture (called PACE) to model embodied and co-creative behavior between humans and embodied intelligent machines. Humans collaborate creatively (co-create) in many settings—for example, dancing with a partner, playing pretend with a child, or brainstorming in an engineering design meeting. As common as this experience is, it rarely defines our interactions with intelligent machines. This project explores improvisational, collaborative, and co-creative dance as a form of non-contact physical interaction between co-creative partners. The project is based upon creative sensemaking theory, which casts the creative practice as a dynamic social process in which individuals alternate between different cognitive states as they make sense of and respond to the actions of others within new environments. This project will promote the progress of science by studying human co-creative practice between advanced dancers to better understand human co-creativity in action and to inform the design and development of co-creative AI technology. The project team will use that understanding to develop AI agents that can co-create with humans. Project outcomes will include novel machine algorithms capable of replicating aspects of human-human co-creativity, as well as public performances of human dancers engaging in co-creative dance with the embodied machine intelligence. This work will inform the long-term development of more effective interfaces in a wide range of applications that involve integrated mind, machine, and motor function—such as physical therapy, design brainstorming, or future experiences with robots in the home. This project also supports K-12, undergraduate, graduate, and public education through outreach and mentorship, and via public human-AI performances.The goal of this project is to develop a modular, reusable system for building embodied co-creative AI. The project team will use contemporary dance as an application domain, as its practitioners are formally trained in exploring, expressing, and collaborating through embodied non-contact physical interactions. The project team will first conduct a qualitative analysis of video and gesture data from human dancer dyads in improvisation sessions. This analysis will be focused on understanding how dancers build understanding through interaction with each other and the environment (i.e. through a process called participatory sensemaking). The team will then formalize ther findings within an architecture for creating virtual, embodied agents that can sense, learn, and generate movement during improvisation. To demonstrate and refine this architecture, the team will develop a co-creative AI that approaches expert-level participatory sensemaking in contemporary dance, and train this agent to create a curated improvisational partner. The agent will be evaluated in rehearsal and in performance. The main contributions of this project will be a) the first open-access annotated dyadic movement dataset; b) a better understanding of how human dancers co-create; c) a interface for dancers to train and improvise with AI; and d) an architecture for making co-creative AI in motor-related domains based on empirical studies of co-creativity.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.