Per Garment Capture and Synthesis for Real-time Virtual Try-on
Per Garment Capture and Synthesis for Real-time Virtual Try-on
复制标题
DOI：
10.1145/3472749.3474762
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
T. Chong;I-Chao Shen;Nobuyuki Umetani;T. Igarashi
中科院分区：
文献类型：
--
作者：
T. Chong;I-Chao Shen;Nobuyuki Umetani;T. Igarashi
虚拟试穿是计算机图形和人机交互的一个很有前途的应用，可以产生深远的现实世界的影响，特别是在这一流行病。现有的基于图像的作品试图从目标服装的单个图像合成试穿图像，但它固有地限制了对可能的交互做出反应的能力。很难再现由姿势和身体尺寸变化以及用手拉动和拉伸服装引起的皱纹变化。在本文中，我们提出了一种替代方案，每个服装捕获和合成工作流程，通过使用许多系统捕获的图像训练模型来处理这种丰富的交互。我们的工作流程由两部分组成：服装捕捉和服装人物图像合成。我们设计了一个驱动的人体模型和一个有效的捕获过程，收集不同的身体尺寸和姿势下的目标服装的详细变形。此外，我们建议使用定制设计的测量服装，我们捕获的测量服装和目标服装的配对图像。然后，我们使用深度图像到图像转换来学习测量服装和目标服装之间的映射。然后，顾客可以在网上购物期间交互地试穿目标服装。拟议的工作流程需要一定的体力劳动，但我们认为，考虑到零售商已经支付了大量成本，聘请专业摄影师和模特，造型师和编辑拍摄照片进行推广，成本是可以接受的。我们的方法可以消除雇用这些昂贵的专业人员的需要。我们评估了所提出的系统的有效性与消融研究和质量比较与以前的虚拟试穿方法。我们进行用户研究，以显示我们有前途的虚拟试穿性能。此外，我们还表明，我们使用我们的方法在视频会议中改变虚拟服装。最后，我们提供收集的数据集作为由各种视角、身体姿势和尺寸参数化的布料数据集。
Virtual try-on is a promising application of computer graphics and human computer interaction that can have a profound real-world impact especially during this pandemic. Existing image-based works try to synthesize a try-on image from a single image of a target garment, but it inherently limits the ability to react to possible interactions. It is difficult to reproduce the change of wrinkles caused by pose and body size change, as well as pulling and stretching of the garment by hand. In this paper, we propose an alternative per garment capture and synthesis workflow to handle such rich interactions by training the model with many systematically captured images. Our workflow is composed of two parts: garment capturing and clothed person image synthesis. We designed an actuated mannequin and an efficient capturing process that collects the detailed deformations of the target garments under diverse body sizes and poses. Furthermore, we proposed to use a custom-designed measurement garment, and we captured paired images of the measurement garment and the target garments. We then learn a mapping between the measurement garment and the target garments using deep image-to-image translation. The customer can then try on the target garments interactively during online shopping. The proposed workflow requires certain manual labor, but we believe that the cost is acceptable given that the retailers are already paying significant costs for hiring professional photographers and models, stylists, and editors to take photographs for promotion. Our method can remove the need of hiring these costly professionals. We evaluated the effectiveness of the proposed system with ablation studies and quality comparison with previous virtual try-on methods. We perform a user study to show our promising virtual try-on performances. Moreover, we also demonstrate that we use our method for changing virtual costumes in video conferences. Finally, we provide the collected dataset as the cloth dataset parameterized by various viewing angles, body poses, and sizes.