Muscles in Action
Muscles in Action
复制标题
DOI：
10.1109/iccv51070.2023.02019
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Mia Chiquier;Carl Vondrick
中科院分区：
文献类型：
--
作者：
Mia Chiquier;Carl Vondrick
人类的运动是由我们的肌肉创造和约束的。我们迈出了构建计算机视觉方法的第一步，这些方法代表了导致运动的内部肌肉活动。我们提出了一个新的数据集，肌肉在行动（MIA），学习将肌肉活动到人体运动表示。该数据集由10名受试者进行各种运动的12.5小时同步视频和表面肌电图（sEMG）数据组成。使用这个数据集，我们学习了一种双向表示，可以从视频中预测肌肉激活，反过来，从肌肉激活中重建运动。我们评估我们的模型在分布的科目和练习，以及在分布的科目和练习。我们展示了如何在建模两种方式的进步，共同可以作为肌肉一致的运动生成条件。将肌肉放入计算机视觉系统将使虚拟人的模型更加丰富，并应用于体育，健身和AR/VR。
Human motion is created by, and constrained by, our muscles. We take a first step at building computer vision methods that represent the internal muscle activity that causes motion. We present a new dataset, Muscles in Action (MIA), to learn to incorporate muscle activity into human motion representations. The dataset consists of 12.5 hours of synchronized video and surface electromyography (sEMG) data of 10 subjects performing various exercises. Using this dataset, we learn a bidirectional representation that predicts muscle activation from video, and conversely, reconstructs motion from muscle activation. We evaluate our model on in-distribution subjects and exercises, as well as on out-of-distribution subjects and exercises. We demonstrate how advances in modeling both modalities jointly can serve as conditioning for muscularly consistent motion generation. Putting muscles into computer vision systems will enable richer models of virtual humans, with applications in sports, fitness, and AR/VR.