Animating Humans from Static Images via an Entirely Image-Based Approach

通过完全基于图像的方法从静态图像中赋予人类动画

• 批准号: EP/F066473/1
• 负责人: Feng Dong
• 金额: $ 10.2万
• 依托单位: University of Bedfordshire
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2008
• 资助国家: 英国
• 起止时间: 2008 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FF066473%2F1
• 关键词: Animating; Humans; Static; Images; via

Images/videos have a promising future for figure animation, an entirely image/video-based approach would allow us to achieve high realism by directly utilising real images/videos. Unfortunately, making effective use of real world images/videos is not a simple task and it is very difficult to reconstruct 3D arbitrary views for human motion. Currently, the Image/Video Based Rendering(IVBR) achieves this by using either captured or adapted generic human geometry from 3D scanners or multi-cameras, which involves costly resources.In fact, the human brain has a strong in-built capacity to imagine motion from static objects. Given a few images of a human motion, we can easily interpret them by envisaging a virtual movement in our mind, without the need of any geometric information. However, existing computing technology still largely falls short of such a capability. Motivated by this observation, the proposed research is designed to take a highly speculative adventure which will explore novel techniques to equip computers with such an ability. Generally speaking, we shall look into the feasibility of making human characters alive from their static images, allowing arbitrary views of their movement to be directly reconstructed from a few key images without requiring their geometric models. While we target humans here, the methodology examined can be applicable to a broad range of articulated/non-articulated subjects. It will go beyond the current form of IVBR, which was mainly designed for objects with fixed shapes, and will aim to achieve what is traditionally feasible only with the assistance of geometric models. It could lead to an alternative that is fundamentally different from all current techniques.This feasibility study will concentrate only on the most fundamental issue of the entirely image based approach , which is the View Reconstruction for Humans (VRH), i.e. whether we can create images of a human movement under arbitrary viewpoints just from a few static images. To test the idea without losing generality, many datasets involved in our experiment will be created by computers. Once a solution to VRH is found, it will open the door to further investigation using real captured images for the training and also to work on other important issues concerning control, data organization & compression, image compositions, hair and cloth motion, etc., in follow-on projects. To allow for the completion of this feasibility study in a short period, we have designed a detailed research route. A learning-based approach will be taken to build statistical models for image sequences of human motion through training from existing examples. Subsequently, such models will be used to construct new sequences of human motion. While there are many potential ways to provide effective user controls, in order to stay focused on VRH, we shall take the most straightforward control strategy, which will use a selected number of images to indicate the key postures of the actor over the time. This is analogous to the key-frame control strategy that is widely adopted in animation.This research also has strong commercial potential in a broad range of entertainment-related businesses in areas such as image/video editing, computer games, the film industry, etc. They have a major presence in the UK and generate significant global income. It will be actively invovled by our industrial contacts at Antics Technologies and Cinesite. Cinesite is one of the largest companies in the production of computer visual effects and post production in the world, while Antics Technologies provides revolutionary software for full computer animation and has world-wide customers. They have recognized the potential market values of this research and will provide strong support through consultancy, evaluation and exploitation. Antics will provide their latest animation software release for this research at no cost.

图像/视频对于人物动画来说有着广阔的前景，完全基于图像/视频的方法将使我们能够通过直接利用真实的图像/视频来实现高度的真实感。不幸的是，有效利用现实世界的图像/视频并不是一项简单的任务，并且重建人体运动的 3D 任意视图也非常困难。目前，基于图像/视频的渲染（IVBR）通过使用从 3D 扫描仪或多相机捕获或改编的通用人体几何形状来实现这一目标，这涉及昂贵的资源。事实上，人脑具有强大的内在能力来想象静态物体的运动。给定一些人体运动的图像，我们可以通过在脑海中想象虚拟运动来轻松解释它们，而不需要任何几何信息。然而，现有的计算技术仍然很大程度上缺乏这种能力。受这一观察的启发，拟议的研究旨在进行一次高度推测性的冒险，探索为计算机配备这种能力的新技术。一般来说，我们将研究从静态图像中使人物角色变得生动的可行性，允许从一些关键图像直接重建其运动的任意视图，而不需要他们的几何模型。虽然我们在这里针对的是人类，但所研究的方法可以适用于广泛的明确/非明确主题。它将超越目前的IVBR形式（主要针对具有固定形状的物体而设计），并将旨在实现传统上仅在几何模型的帮助下才可行的目标。它可能会带来一种与当前所有技术根本不同的替代方案。这项可行性研究将仅集中于完全基于图像的方法的最基本问题，即人类视图重建（VRH），即我们是否可以仅从一些静态图像在任意视点下创建人体运动的图像。为了在不失普遍性的情况下测试这个想法，我们实验中涉及的许多数据集将由计算机创建。一旦找到 VRH 的解决方案，它将为使用真实捕获的图像进行进一步研究打开大门，并在后续项目中解决有关控制、数据组织和压缩、图像合成、头发和布料运动等其他重要问题。为了能够在短时间内完成本次可行性研究，我们设计了详细的研究路线。将采用基于学习的方法，通过现有示例的训练来构建人体运动图像序列的统计模型。随后，此类模型将用于构建新的人体运动序列。虽然有许多潜在的方法可以提供有效的用户控制，但为了保持对 VRH 的关注，我们将采取最直接的控制策略，即使用选定数量的图像来指示演员随时间变化的关键姿势。这类似于动画中广泛采用的关键帧控制策略。这项研究在图像/视频编辑、电脑游戏、电影业等领域的广泛娱乐相关业务中也具有强大的商业潜力。它们在英国占有重要地位，并在全球产生可观的收入。我们在 Antics Technologies 和 Cinesite 的行业联系人将积极参与其中。 Cinesite 是世界上最大的计算机视觉效果和后期制作公司之一，而 Antics Technologies 则为完整的计算机动画提供革命性的软件，并拥有全球客户。他们已经认识到这项研究的潜在市场价值，并将通过咨询、评估和开发提供强有力的支持。 Antics 将免费为这项研究提供最新的动画软件版本。

项目成果

Laplacian group sparse modeling of human actions

人类行为的拉普拉斯群稀疏建模

• DOI: 10.1016/j.patcog.2014.02.007
• 发表时间: 2014-08
• 期刊: Pattern Recognition
• 影响因子: 8
• 作者: Yang, Hao;Jiao, L. C.;Yang, Yang;Dong, Feng
• 通讯作者: Dong, Feng