Reinforcement Learning for Logic Recipe Generation: Bridging Gaps From Images to Plans
Reinforcement Learning for Logic Recipe Generation: Bridging Gaps From Images to Plans
复制标题
用于逻辑菜谱生成的强化学习：弥合图像与计划之间的差距
DOI：
10.1109/tmm.2021.3050090
复制
发表时间：
2021-01
影响因子：
7.3
通讯作者：
Duan Peng
中科院分区：
文献类型：
--
作者：
Zhang Mengyang;Tian Guohui;Zhang Ying;Duan Peng
It is a challenging task to produce recipes from images, due to the difficulty in bridging the gap from intuitive, static images to sequential, dynamic recipes. In this paper, we propose a novel recipe generation system for producing effective recipes from images. As medium steps, ingredient generation is introduced to guide recipe generation in our system. With potential information in ingredient lists, ingredient selection and ingredient sequence, the system is taught to generate effective recipes. For information representation, a hierarchical attention mechanism is designed to extract effective features for ingredient production and recipe generation. In order to guarantee the comprehensiveness and logic in recipes, a specific and explicit criterion around ingredients is designed under the framework of reinforcement learning. In ingredient generation, the system is required to generate ingredients with correct sequence in cooking procedures. And in recipe generation, ingredients in recipes are required to be consistent with produced ingredients. In experiments, the proposed method is compared with state-of-the-art methods to evaluate the feasibility. The results indicate that the proposed system achieves a better performance than other methods on both aspects of producing proper ingredients and effective recipes.
