Animating Humans from Static Images via an Entirely Image-Based Approach

图像/视频在图形动画方面有着光明的未来，完全基于图像/视频的方法将允许我们通过直接利用真实图像/视频来实现高真实感。不幸的是，有效地利用真实世界的图像/视频并不是一项简单的任务，并且很难重建人体运动的3D任意视图。目前，基于图像/视频的绘制(IVBR)通过使用从3D扫描仪或多摄像机捕获或改编的通用人体几何来实现这一点，这涉及昂贵的资源。事实上，人脑具有很强的内置能力来从静态对象想象运动。给出一些人体运动的图像，我们可以很容易地通过在我们的脑海中想象一个虚拟的运动来解释它们，而不需要任何几何信息。然而，现有的计算技术在很大程度上仍然缺乏这样的能力。在这一观察的推动下，这项拟议的研究旨在进行一次高度投机性的冒险，探索使计算机具备这种能力的新技术。一般而言，我们将研究从静态图像中使人类角色栩栩如生的可行性，允许从几个关键图像直接重建其运动的任意视图，而不需要他们的几何模型。虽然我们在这里以人类为目标，但所研究的方法可以适用于广泛的有关节/无关节的主题。它将超越目前的IVBR形式，后者主要是为具有固定形状的物体设计的，它的目标是实现传统上只有在几何模型的帮助下才能实现的东西。这一可行性研究将集中在完全基于图像的方法中最基本的问题上，即人类的视觉重建(VRH)，即我们是否可以仅从几幅静态图像在任意视点下创建人体运动的图像。为了在不失去通用性的情况下测试这个想法，我们的实验中涉及的许多数据集将由计算机创建。一旦找到了VRH的解决方案，它将打开进一步调查的大门，使用真实捕获的图像进行培训，并在后续项目中研究其他重要问题，如控制、数据组织和压缩、图像合成、头发和布料运动等。为了在短时间内完成这项可行性研究，我们设计了一条详细的研究路线。采用基于学习的方法，通过训练已有的样本，建立人体运动图像序列的统计模型。随后，这种模型将被用来构建新的人体运动序列。虽然有许多可能的方法来提供有效的用户控制，为了保持对VRH的关注，我们将采取最直接的控制策略，它将使用选定的图像数量来指示演员在一段时间内的关键姿势。这类似于动画中广泛采用的关键帧控制策略。这项研究在图像/视频编辑、计算机游戏、电影工业等领域的广泛娱乐相关业务中也具有强大的商业潜力。它们在英国有很大的影响力，并在全球范围内创造了可观的收入。我们在Antics Technologies和Cinesite的行业联系人将积极参与其中。Cinesite是世界上最大的计算机视觉效果制作和后期制作公司之一，而Antics Technologies为全电脑动画提供革命性的软件，并拥有世界各地的客户。他们已经认识到这项研究的潜在市场价值，并将通过咨询、评估和开发提供强有力的支持。滑稽演员将免费为这项研究提供最新的动画软件版本。

Images/videos have a promising future for figure animation, an entirely image/video-based approach would allow us to achieve high realism by directly utilising real images/videos. Unfortunately, making effective use of real world images/videos is not a simple task and it is very difficult to reconstruct 3D arbitrary views for human motion. Currently, the Image/Video Based Rendering(IVBR) achieves this by using either captured or adapted generic human geometry from 3D scanners or multi-cameras, which involves costly resources.In fact, the human brain has a strong in-built capacity to imagine motion from static objects. Given a few images of a human motion, we can easily interpret them by envisaging a virtual movement in our mind, without the need of any geometric information. However, existing computing technology still largely falls short of such a capability. Motivated by this observation, the proposed research is designed to take a highly speculative adventure which will explore novel techniques to equip computers with such an ability. Generally speaking, we shall look into the feasibility of making human characters alive from their static images, allowing arbitrary views of their movement to be directly reconstructed from a few key images without requiring their geometric models. While we target humans here, the methodology examined can be applicable to a broad range of articulated/non-articulated subjects. It will go beyond the current form of IVBR, which was mainly designed for objects with fixed shapes, and will aim to achieve what is traditionally feasible only with the assistance of geometric models. It could lead to an alternative that is fundamentally different from all current techniques.This feasibility study will concentrate only on the most fundamental issue of the entirely image based approach , which is the View Reconstruction for Humans (VRH), i.e. whether we can create images of a human movement under arbitrary viewpoints just from a few static images. To test the idea without losing generality, many datasets involved in our experiment will be created by computers. Once a solution to VRH is found, it will open the door to further investigation using real captured images for the training and also to work on other important issues concerning control, data organization & compression, image compositions, hair and cloth motion, etc., in follow-on projects. To allow for the completion of this feasibility study in a short period, we have designed a detailed research route. A learning-based approach will be taken to build statistical models for image sequences of human motion through training from existing examples. Subsequently, such models will be used to construct new sequences of human motion. While there are many potential ways to provide effective user controls, in order to stay focused on VRH, we shall take the most straightforward control strategy, which will use a selected number of images to indicate the key postures of the actor over the time. This is analogous to the key-frame control strategy that is widely adopted in animation.This research also has strong commercial potential in a broad range of entertainment-related businesses in areas such as image/video editing, computer games, the film industry, etc. They have a major presence in the UK and generate significant global income. It will be actively invovled by our industrial contacts at Antics Technologies and Cinesite. Cinesite is one of the largest companies in the production of computer visual effects and post production in the world, while Antics Technologies provides revolutionary software for full computer animation and has world-wide customers. They have recognized the potential market values of this research and will provide strong support through consultancy, evaluation and exploitation. Antics will provide their latest animation software release for this research at no cost.