Joint 3D Human Shape Recovery and Pose Estimation from a Single Image with Bilayer Graph
Joint 3D Human Shape Recovery and Pose Estimation from a Single Image with Bilayer Graph
复制标题
DOI：
10.1109/3dv53792.2021.00060
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Xin Yu;J. Baar;Siheng Chen
中科院分区：
文献类型：
--
作者：
Xin Yu;J. Baar;Siheng Chen
从图像估计3D人体形状和姿势的能力在许多情况下是有用的。最近的方法使用图卷积网络进行了探索，并取得了可喜的成果。3D形状由网格（一种无向图）表示，这一事实使得图卷积网络很自然地适合这个问题。然而，图卷积网络具有有限的表示能力，来自图中节点的信息被传递给连接的邻居，并且信息的传播需要连续的图卷积。为了克服这个限制，我们提出了一个双尺度图的方法。我们使用一个粗糙的图，从一个密集的图，估计人的三维姿态，和密集的图估计的三维形状。与密集图相比，粗糙图中的信息可以传播更长的距离。此外，姿态信息可以指导恢复局部形状细节，反之亦然。我们认识到粗糙和密集之间的连接本身就是一个图，并引入图融合块来交换不同尺度的图之间的信息。我们端到端地训练我们的模型，并证明我们可以为几个评估数据集实现最先进的结果。代码可在以下链接https://github.com/yuxwind/BiGraphBody上获得。
The ability to estimate the 3D human shape and pose from images can be useful in many contexts. Recent approaches have explored using graph convolutional networks and achieved promising results. The fact that the 3D shape is represented by a mesh, an undirected graph, makes graph convolutional networks a natural fit for this problem. However, graph convolutional networks have limited representation power Information from nodes in the graph is passed to connected neighbors, and propagation of information requires successive graph convolutions. To overcome this limitation, we propose a dual-scale graph approach. We use a coarse graph, derived from a dense graph, to estimate the human’s 3D pose, and the dense graph to estimate the 3D shape. Information in coarse graphs can be propagated over longer distances compared to dense graphs. In addition, information about pose can guide to recover local shape detail and vice versa. We recognize that the connection between coarse and dense is itself a graph, and introduce graph fusion blocks to exchange information between graphs with different scales. We train our model end-to-end and show that we can achieve state-of-the-art results for several evaluation datasets. The code is available at the following link, https://github.com/yuxwind/BiGraphBody.