EAGER: Efficient control and transmission of digital puppetry

点击翻译按钮获取中文摘要

Digital puppetry refers to the interactive control of virtual characters. This approach can be useful in any virtual experience where a synthetic character?s interactivity must surpass the current capabilities of AI. Examples of this include teacher training; projecting remote docents into museums, science centers and networked virtual environments; and preparing soldiers for their first tours of duty in foreign lands. The objective of this project is to improve the quality and breadth of experiences that can be provided by the puppeteering paradigm, especially when the puppeteer is working from a remote location such as his or her home and the experience is taking place in a location with limited network connectivity. The intent is to reduce latency and bandwidth requirements; cost and complexity of experience creation, capture and delivery; and cognitive load on puppeteers ? while still providing support for complex behaviors. This exploratory research project focuses on micro-poses that can be recognized and assembled into more complex actions. The PI seeks to rapidly identify these poses via the fusion of multiple inexpensive sources of sensor data and the physical constraints appropriate for our virtual characters. The PI approaches the tension between precision and network demand by using these micro-poses to reduce cost, footprint of the puppetry motion capture, and networking demands, while simultaneously increasing the effectiveness and accessibility of the puppeteering paradigm. The focus on accessibility leads to the development of techniques that can be run at interactive rates on in-home tabletop systems, facilitating connections between mentors and mentees across the globe.