Modeling Diverse, Personalized and Expressive Animations for Virtual Characters through Motion Capture, Synthesis and Perception

角色动画在游戏开发、机器人、虚拟现实（VR）和增强现实（AR）应用中为虚拟角色提供运动方面发挥着关键作用。尽管在建模运动是自然和逼真的大的努力，以前的研究一直集中在通用的运动内容学习，其中每个人如何执行的内容的风格特征在很大程度上被忽略。在没有个性化风格的情况下，不同的虚拟角色在需要相同的运动内容时以相同的方式移动，这对于创建多样化的虚拟世界来说是远远不能令人满意的。风格运动的一个主要挑战是缺乏大规模的运动风格数据库，因此没有足够的知识如何有效地建模和转移风格。该建议侧重于运动风格的学习，合成和迁移。从长远来看，我们的目标是找到最终的解决方案，以生成风格化的运动与变化，以匹配在现实世界中的多样性。短期目标是：我们将首先利用动作捕捉技术建立大规模的动作风格数据库; 2从这些数据中，我们将开发出学习有效动作表示的方法，并探索出有条件地生成具有所需风格的动作的生成模型; 3我们将进一步发现风格转换模型，以在保持原始动作内容的同时编辑风格。 在五年的时间里，我们将专门捕捉、学习和建模三种风格特征：属于不同人群的人口统计风格，例如年龄、性别和种族;不同个体的个性、体型所产生的个性化风格;在不同场景下同一个体表现出不同情绪和身体状态的表达风格。我们将建立我们的数据库，以涵盖这些风格的变化，公布数据库的开放获取，并向公众提供标签，文件和技术支持。研究成果和源代码将公布，解决提取风格特征，以可控的方式生成运动风格，并将风格转换为新的运动。在五年任期之后，我们将继续向我们的数据库添加更多的风格，以建立更广泛的运动风格模型。我们的数据库可以直接用于动画在AR/VR和游戏场景中的各种角色，它也方便了其他研究人员建模的运动风格，并刺激心理学，运动学和艺术等跨学科的研究在运动风格建模的研究成果支持智能应用，如动作识别，风格识别和运动输入的人识别。风格合成和转移技术还可以广泛应用于游戏、娱乐行业，AR/VR应用于教育、媒体、社交网络。通过创造真实地体现真实的世界中的人的虚拟角色，这项工作可以促进虚拟世界的多样性，公平性和包容性。

Character animation plays a key role in delivering motions for virtual characters in game development, robotics, Virtual Reality (VR) and Augmented Reality (AR) applications. Despite the large effort in modeling motions to be natural and realistic, previous research has been focused on generic motion content learning where stylistic features of how each individual performs the content are largely ignored. Without individual styles, different virtual characters move the same way when the same motion content is needed, which is far from satisfactory to create a diverse virtual world. One major challenge to stylize motions is the lack of large scale motion style databases, and thus no sufficient knowledge of how to effectively model and transfer styles. This proposal focuses on motion style learning, synthesis and transfer. In the long term, our goal is to find ultimate solutions to generate stylized motions with variations that match the diversity in the real-world. The short-term objectives are: we will first establish large scale motion style databases through motion capture technique; from the data, we develop methods to learn effective motion representations and explore generative models to conditionally generate motions with desired styles; we will further discover style transfer models to edit styles while keeping the original motion content. In the five-year period, we will specifically capture, learn and model three stylistic features: demographic styles belonging to different groups of people, e.g. age, gender, and race; personalized styles resulting from personalities, body build for different individuals; expressive styles demonstrating varied emotional and physical states for the same individual under different scenarios. We will set up our databases to cover these style variations, publicize the database for open access, and provide labelling, documentation and technical support to the public. Research findings and source code will be published, solving problems of extracting style features, generating motion styles in a controllable manner, and transferring styles to novel motions. Beyond the five-year term, we will continue adding more styles to our databases, to model a broader picture of motion styles. Our database can directly be used in animating diverse characters in AR/VR and game scenes, it also facilitates other researchers to model motion styles, and stimulates interdisciplinary research in psychology, kinesiology, and art etc. Research findings in motion style modeling supports intelligent applications such as action recognition, style recognition and person identification from motion input. Style synthesis and transfer technology can also be widely applied in the game, entertainment industry, AR/VR applications in education, media, and social networks. By creating virtual characters that authentically embody people in the real world, this work can promote Diversity, Equity and Inclusion of the virtual world.