Modeling Diverse, Personalized and Expressive Animations for Virtual Characters through Motion Capture, Synthesis and Perception

通过动作捕捉、合成和感知为虚拟角色建模多样化、个性化和富有表现力的动画

• 批准号: RGPIN-2022-04920
• 负责人: Wang, Yingying
• 金额: $ 2.11万
• 依托单位: McMaster University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750440
• 关键词: Modeling; Diverse; Personalized; Expressive; Animations

Character animation plays a key role in delivering motions for virtual characters in game development, robotics, Virtual Reality (VR) and Augmented Reality (AR) applications. Despite the large effort in modeling motions to be natural and realistic, previous research has been focused on generic motion content learning where stylistic features of how each individual performs the content are largely ignored. Without individual styles, different virtual characters move the same way when the same motion content is needed, which is far from satisfactory to create a diverse virtual world. One major challenge to stylize motions is the lack of large scale motion style databases, and thus no sufficient knowledge of how to effectively model and transfer styles. This proposal focuses on motion style learning, synthesis and transfer. In the long term, our goal is to find ultimate solutions to generate stylized motions with variations that match the diversity in the real-world. The short-term objectives are: we will first establish large scale motion style databases through motion capture technique; from the data, we develop methods to learn effective motion representations and explore generative models to conditionally generate motions with desired styles; we will further discover style transfer models to edit styles while keeping the original motion content. In the five-year period, we will specifically capture, learn and model three stylistic features: demographic styles belonging to different groups of people, e.g. age, gender, and race; personalized styles resulting from personalities, body build for different individuals; expressive styles demonstrating varied emotional and physical states for the same individual under different scenarios. We will set up our databases to cover these style variations, publicize the database for open access, and provide labelling, documentation and technical support to the public. Research findings and source code will be published, solving problems of extracting style features, generating motion styles in a controllable manner, and transferring styles to novel motions. Beyond the five-year term, we will continue adding more styles to our databases, to model a broader picture of motion styles. Our database can directly be used in animating diverse characters in AR/VR and game scenes, it also facilitates other researchers to model motion styles, and stimulates interdisciplinary research in psychology, kinesiology, and art etc. Research findings in motion style modeling supports intelligent applications such as action recognition, style recognition and person identification from motion input. Style synthesis and transfer technology can also be widely applied in the game, entertainment industry, AR/VR applications in education, media, and social networks. By creating virtual characters that authentically embody people in the real world, this work can promote Diversity, Equity and Inclusion of the virtual world.

角色动画在游戏开发、机器人、虚拟现实（VR）和增强现实（AR）应用中为虚拟角色提供动作方面发挥着关键作用。尽管在运动建模方面做出了很大的努力，使其自然和逼真，但以前的研究主要集中在一般的运动内容学习上，其中每个个体如何执行内容的风格特征在很大程度上被忽略了。如果没有个性化的风格，不同的虚拟角色在需要相同的动作内容时就会以相同的方式移动，这远远不能满足创造多样化的虚拟世界。对运动进行风格化的一个主要挑战是缺乏大规模的运动风格数据库，因此没有足够的知识来有效地建模和转移风格。本研究的重点是动作风格的学习、合成和迁移。从长远来看，我们的目标是找到最终的解决方案，以生成与现实世界多样性相匹配的程式化运动。短期目标是：首先通过动作捕捉技术建立大规模的动作风格数据库；从数据中，我们开发了学习有效运动表示的方法，并探索生成模型，以有条件地生成具有所需风格的运动；我们将进一步发现风格转移模型来编辑风格，同时保持原始的运动内容。在五年期间，我们将专门捕捉、学习和建模三种风格特征：属于不同人群的人口统计风格，例如年龄、性别和种族；因个性、体型不同而产生的个性化风格；同一个体在不同情境下表现出不同的情绪和身体状态的表达风格。我们会建立资料库以涵盖这些不同的风格，并公开资料库以供开放使用，并向公众提供标签、文件和技术支援。发表研究成果及源代码，解决风格特征提取、运动风格可控生成、风格向新动作转化等问题。在五年的期限之后，我们将继续向我们的数据库中添加更多的样式，以建立更广泛的运动样式模型。我们的数据库可以直接用于AR/VR和游戏场景中的各种角色动画，也方便其他研究人员建模运动风格，并促进心理学，运动机能学和艺术等跨学科研究。运动风格建模的研究成果支持智能应用，如动作识别、风格识别和从运动输入识别人。风格合成和转移技术也可以广泛应用于游戏，娱乐行业，教育，媒体和社交网络中的AR/VR应用。通过创造真实体现现实世界中的人的虚拟角色，这项工作可以促进虚拟世界的多样性，公平性和包容性。