Modeling Diverse, Personalized and Expressive Animations for Virtual Characters through Motion Capture, Synthesis and Perception

点击翻译按钮获取中文摘要

Character animation plays a key role in delivering motions for virtual characters in game development, robotics, Virtual Reality (VR) and Augmented Reality (AR) applications. Despite the large effort in modeling motions to be natural and realistic, previous research has been focused on generic motion content learning where stylistic features of how each individual performs the content are largely ignored. Without individual styles, different virtual characters move the same way when the same motion content is needed, which is far from satisfactory to create a diverse virtual world. One major challenge to stylize motions is the lack of large scale motion style databases, and thus no sufficient knowledge of how to effectively model and transfer styles. This proposal focuses on motion style learning, synthesis and transfer. In the long term, our goal is to find ultimate solutions to generate stylized motions with variations that match the diversity in the real-world. The short-term objectives are: we will first establish large scale motion style databases through motion capture technique; from the data, we develop methods to learn effective motion representations and explore generative models to conditionally generate motions with desired styles; we will further discover style transfer models to edit styles while keeping the original motion content. In the five-year period, we will specifically capture, learn and model three stylistic features: demographic styles belonging to different groups of people, e.g. age, gender, and race; personalized styles resulting from personalities, body build for different individuals; expressive styles demonstrating varied emotional and physical states for the same individual under different scenarios. We will set up our databases to cover these style variations, publicize the database for open access, and provide labelling, documentation and technical support to the public. Research findings and source code will be published, solving problems of extracting style features, generating motion styles in a controllable manner, and transferring styles to novel motions. Beyond the five-year term, we will continue adding more styles to our databases, to model a broader picture of motion styles. Our database can directly be used in animating diverse characters in AR/VR and game scenes, it also facilitates other researchers to model motion styles, and stimulates interdisciplinary research in psychology, kinesiology, and art etc. Research findings in motion style modeling supports intelligent applications such as action recognition, style recognition and person identification from motion input. Style synthesis and transfer technology can also be widely applied in the game, entertainment industry, AR/VR applications in education, media, and social networks. By creating virtual characters that authentically embody people in the real world, this work can promote Diversity, Equity and Inclusion of the virtual world.