Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows
Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows
复制标题
DOI：
10.48550/arxiv.2209.05840
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Keisuke Shirai;Atsushi Hashimoto;Taichi Nishimura;Hirotaka Kameko;Shuhei Kurita;Y. Ushiku;Shinsuke Mori
中科院分区：
文献类型：
--
作者：
Keisuke Shirai;Atsushi Hashimoto;Taichi Nishimura;Hirotaka Kameko;Shuhei Kurita;Y. Ushiku;Shinsuke Mori
我们提出了一个名为“视觉食谱流程”的新的多模态数据集，它使我们能够学习食谱文本中每个对象的烹饪动作结果。该数据集由对象状态变化和食谱文本的工作流程组成。状态变化以一对图像表示，而工作流程以食谱流程图表示。我们开发了一个网络界面以降低人工标注成本。该数据集使我们能够尝试各种应用，包括多模态信息检索。
We present a new multimodal dataset called Visual Recipe Flow, which enables us to learn a cooking action result for each object in a recipe text. The dataset consists of object state changes and the workflow of the recipe text. The state change is represented as an image pair, while the workflow is represented as a recipe flow graph. We developed a web interface to reduce human annotation costs. The dataset allows us to try various applications, including multimodal information retrieval.