Multilinear Modelling of Faces and Expressions
Multilinear Modelling of Faces and Expressions
复制标题
DOI：
10.1109/tpami.2020.2986496
复制
发表时间：
2021-10-01
影响因子：
23.6
通讯作者：
Ostermann, Joern
中科院分区：
文献类型：
--
作者：
Grasshof, Stella;Ackermann, Hanno;Ostermann, Joern
在这项工作中，我们提出了一种新的通用的3D多线性统计人脸模型，该模型基于3D人脸扫描的张量分解，将形状分解为人和表情的子空间。对表达式子空间的研究揭示了固有的低维子结构，并进一步揭示了星形结构。这要归功于两个新发现。(1)增加一种情绪的强度在子空间中近似形成一条线性轨迹。(2)所有这些轨迹都在一个点上相交--不是像几乎所有以前的作品所假设的那样，在中性的表达上--而是在一种冷漠的表达上。我们利用这些结构发现，通过以冷静点为中心的四阶矩张量对表达式子空间进行重新参数化。我们提出了一种基于单个或多个2D投影的3D人脸重建方法，该方法假设一个未标定的投影摄像机模型。由透视投影引起的非线性可以巧妙地包含在模型中。该算法令人信服地分离了人和表情的子空间，并为各种人脸的表情提供了灵活、自然的建模。在独立人脸上的应用表明，该方法可以在不同的人和表情之间进行变形，而不会产生强烈的变形。
In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works-but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.