自动编舞和作曲的对偶学习及其应用相关技术研究

目前，虽然科技和艺术的深度融合已被推向历史最高点，但是科技支撑艺术创作的巨大潜力远未真正释放。自动编舞和自动作曲是人工智能支撑艺术创作的典型代表，一直以来作为两个相互独立的研究领域而存在。然而，艺术学和认知神经学的研究均表明，舞蹈和音乐之间存在异质同构关系。基于这一现实依据，本项目在对偶学习框架下研究自动编舞和作曲：利用舞蹈和音乐之间节奏同步、主题一致等约束，将二者作为彼此的监督信号联合协同训练自动编舞和作曲模型。相比已有方法中基于随机游走的合成方式，对偶学习能够通过引入反馈和正则有效提升合成舞蹈和音乐的质量。为将合成的舞蹈和音乐对接至终端大众需求和产业应用，在自动编舞和作曲的基础上研究人机交互引导的舞蹈和音乐风格迁移以及舞蹈动作重定向。本项目的实施有助于降低大众参与舞蹈和音乐创作的技术门槛和难度，助推自动编舞和作曲在影视、动画、游戏、元宇宙数字人等产业的应用，建立舞蹈和音乐创作的新模式。

Currently, although deep integration of science, technology, and art has been pushed to the highest point in history, the great potential of science and technology to support artistic creation is far from being truly released. Automatic choreography and automatic composition are the typical representatives of artificial intelligence supporting artistic creation, and the two fields are always independently studied. However, studies in art and cognitive neuroscience have shown that there exists a heterogeneous isomorphic relationship between dance and music. In view of this practical basis, this project studies automatic choreography and composition under the framework of dual learning: using the constraints of rhythm synchronization and theme consistency between dance and music, dance and music are treated as the mutual supervision signals for jointly and synergistically training an automatic choreography and composition model. Compared with the existing methods that synthetize dance and music in a random walk way, dual learning is able to effectively improve the quality of synthetic music and dance by introducing feedback and regularization. In order to connect the synthesized dance and music to public demand and industrial applications, this project additionally studies human-computer-interaction-guided dance and music style transfer, and dance retargeting based on the studies of automatic choreography and composition. The implementation of this project will help to reduce the technical threshold and difficulty for the public to participate in dance and music creation, boost the application of automatic choreography and composition in film, television, animation, games, and meta humans, and establish a new mode of artistic creation.