Joint Discovery of Object States and Manipulation Actions
Joint Discovery of Object States and Manipulation Actions
复制标题
DOI：
10.1109/iccv.2017.234
复制
发表时间：
2017-02
期刊：
影响因子：
--
通讯作者：
Jean-Baptiste Alayrac;Josef Sivic;I. Laptev;Simon Lacoste-Julien
中科院分区：
文献类型：
--
作者：
Jean-Baptiste Alayrac;Josef Sivic;I. Laptev;Simon Lacoste-Julien
许多人类活动涉及旨在修改对象状态的对象操作。常见状态变化的示例包括满/空瓶子、打开/关闭的门以及连接/分离的车轮。在这项工作中，我们寻求自动发现对象的状态和相关的操纵动作。给定特定任务的一组视频，我们提出了一个联合模型，可以学习识别对象状态并本地化状态修改动作。我们的模型被表述为具有约束的判别聚类成本。我们假设对象状态和操作动作的变化具有一致的时间顺序，并引入新的优化技术来学习模型参数，而无需额外的监督。我们在描述现实生活中对象操作的新视频数据集上成功发现了七个操作动作和相应的对象状态。我们表明，我们的联合公式可以改善通过动作识别来发现对象状态，反之亦然。
Many human activities involve object manipulations aiming to modify the object state. Examples of common state changes include full/empty bottle, open/closed door, and attached/detached car wheel. In this work, we seek to automatically discover the states of objects and the associated manipulation actions. Given a set of videos for a particular task, we propose a joint model that learns to identify object states and to localize state-modifying actions. Our model is formulated as a discriminative clustering cost with constraints. We assume a consistent temporal order for the changes in object states and manipulation actions, and introduce new optimization techniques to learn model parameters without additional supervision. We demonstrate successful discovery of seven manipulation actions and corresponding object states on a new dataset of videos depicting real-life object manipulations. We show that our joint formulation results in an improvement of object state discovery by action recognition and vice versa.