Image-Based Food Calorie Estimation Using Recipe Information
Image-Based Food Calorie Estimation Using Recipe Information
复制标题
DOI：
10.1587/transinf.2017mvp0027
复制
发表时间：
2018-05-01
影响因子：
0.7
通讯作者：
Yanai, Keiji
中科院分区：
文献类型：
--
作者：
Ege, Takumi;Yanai, Keiji
最近，记录日常饮食的手机应用程序引起了人们对自我饮食的关注。然而，大多数应用程序返回的食物卡路里值只是与估计的食物类别相关联，或者需要用户手动指示食物的大致数量。事实上，从食物照片中准确估计食物的热量还没有实现，这是一个尚未解决的问题。然后，在本文中，我们提出通过深度学习同时学习食物卡路里，类别，成分和烹饪方向，从食物照片中估计食物卡路里。由于食物卡路里与食物类别、食材和烹饪方向信息之间存在很强的相关性，我们预计它们的同时训练比独立的单一训练更能提高成绩。为此，我们使用多任务CNN。此外，在本研究中，我们构建了两种数据集，一种是来自Web上日本食谱网站的卡路里注释食谱数据集，另一种是来自美国食谱网站的数据集。在实验中，我们训练了多任务和单任务cnn，并对它们进行了比较。因此，多任务CNN在食物类别估计和食物卡路里估计上都比单任务CNN有更好的表现。对于日本食谱数据集，通过引入多任务CNN，相关系数提高了0.039，而对于美国食谱数据集，相关系数比单任务CNN提高了0.090。此外，我们还发现本文提出的基于多任务CNN的方法优于之前提出的基于搜索的方法。
Recently, mobile applications for recording everyday meals draw much attention for self dietary. However, most of the applications return food calorie values simply associated with the estimated food categories, or need for users to indicate the rough amount of foods manually. In fact, it has not been achieved to estimate food calorie from a food photo with practical accuracy, and it remains an unsolved problem. Then, in this paper, we propose estimating food calorie from a food photo by simultaneous learning of food calories, categories, ingredients and cooking directions using deep learning. Since there exists a strong correlation between food calories and food categories, ingredients and cooking directions information in general, we expect that simultaneous training of them brings performance boosting compared to independent single training. To this end, we use a multi-task CNN. In addition, in this research, we construct two kinds of datasets that is a dataset of calorie-annotated recipe collected from Japanese recipe sites on the Web and a dataset collected from an American recipe site. In the experiments, we trained both multi-task and single-task CNNs, and compared them. As a result, a multi-task CNN achieved the better performance on both food category estimation and food calorie estimation than single-task CNNs. For the Japanese recipe dataset, by introducing a multi-task CNN, 0.039 were improved on the correlation coefficient, while for the American recipe dataset, 0.090 were raised compared to the result by the single-task CNN. In addition, we showed that the proposed multi-task CNN based method outperformed search-based methods proposed before.