Generative Modelling for Sequenatial Human Behaviour

人类连续行为的生成模型

• 批准号: 2130174
• 金额: --
• 依托单位: Imperial College London
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2130174
• 关键词: Generative; Modelling; Sequenatial; Human; Behaviour

The goal of this research is to analyze and model sequential human behaviour such as speech,facial animation and gestures. The first step towards achieving this is to identify the factors thatcontribute and guide human behaviour. Generative models can reveal hidden structures in thedata which are often referred to as the latent representation of the data. As part of my researchI want to focus on Finding ways to disentangle these latent variables in order to gain control overdifferent aspects the generated sequences.I would like to explore recent generative methods and adapt them to handle sequential data.One such model is generative adversarial networks, which uses a discriminating network to drive thelearning of a generating network. This approach has been very successful for generating static dataand its extension to sequences is a very active area of research. Another advantage of exploringthis approach is that it allows the use of multiple discriminator networks that are capable ofsimultaneously capturing various aspects of real human behavioural data.Another goal of this research is to understand the relationship between the signals that make uphuman behaviour because these signals are often linked. A good example of this is the correlationbetween human speech and facial animation. I plan to exploit this relationship to perform speech-driven animation which will greatly reduce the cost of computer generated imagery (CGI).Additionally, I believe that it is important to research methods that model the changes in signalsrather than the signals themselves. Such models may be better suited to capture the dynamics ofmost natural systems, hence I would like to research new network architectures that make theseapproaches possible.Finally as we seek to constantly improve the realism of generated human behavioural data itis also important to find ways to distinguish generated signals from real ones. Generating contentthat is very realistic can have great security implications (i.e. identity theft) and therefore I wouldalso like to explore ways of distinguishing real and generated human behavioural data.This research is in line with the goals of EPSRC in the fields of computer graphics and human-computer interaction since it will enable the fast and efficient generation of animated characters.

这项研究的目标是分析和建模连续的人类行为，如语音，面部动画和手势。实现这一目标的第一步是确定促成和指导人类行为的因素。生成模型可以揭示数据中隐藏的结构，这些结构通常被称为数据的潜在表示。作为我研究的一部分，我想专注于寻找解开这些潜在变量的方法，以便控制生成序列的不同方面。我想探索最新的生成方法，并使其适应处理序列数据。其中一个模型是生成对抗网络，它使用一个判别网络来驱动生成网络的学习。这种方法已经非常成功地产生静态数据和序列的扩展是一个非常活跃的研究领域。探索这种方法的另一个优点是它允许使用多个能够同时捕获真实的人类行为数据的各个方面的神经网络。这项研究的另一个目标是了解使人类行为的信号之间的关系，因为这些信号通常是联系在一起的。一个很好的例子是人类语言和面部动画之间的相关性。我计划利用这种关系来执行语音驱动的动画，这将大大降低计算机生成图像（CGI）的成本。此外，我认为重要的是研究模拟信号变化的方法，而不是信号本身。这样的模型可能更适合捕捉大多数自然系统的动态，因此我想研究新的网络架构，使这些方法成为可能。最后，当我们寻求不断提高生成的人类行为数据的真实性时，找到将生成的信号与真实的信号区分开来的方法也很重要。生成非常逼真的内容可能会产生很大的安全影响（即身份盗窃），因此我也想探索区分真实的和生成的人类行为数据的方法。这项研究符合EPSRC在计算机图形学和人机交互领域的目标，因为它将使动画角色的快速有效生成成为可能。

项目成果

Video-Driven Speech Reconstruction using Generative Adversarial Networks

• DOI: 10.21437/interspeech.2019-1445
• 发表时间: 2019-06
• 作者: Konstantinos Vougioukas;Pingchuan Ma;Stavros Petridis;M. Pantic

Realistic Speech-Driven Facial Animation with GANs

• DOI: 10.1007/s11263-019-01251-8
• 发表时间: 2019-06
• 期刊: International Journal of Computer Vision
• 影响因子: 19.5
• 作者: Konstantinos Vougioukas;Stavros Petridis;M. Pantic

Speech-Driven Facial Animation Using Polynomial Fusion of Features

• DOI: 10.1109/icassp40776.2020.9054469
• 发表时间: 2019-12
• 期刊: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Triantafyllos Kefalas;Konstantinos Vougioukas;Yannis Panagakis;Stavros Petridis;Jean Kossaifi;M. Pantic

Visually Guided Self Supervised Learning of Speech Representations

语音表示的视觉引导自监督学习

• DOI: 10.1109/icassp40776.2020.9053415
• 发表时间: 2020
• 作者: Shukla A