Ukemochi: A Video See-through Food Overlay System for Eating Experience in the Metaverse
Ukemochi: A Video See-through Food Overlay System for Eating Experience in the Metaverse
复制标题
Ukemochi：用于虚拟宇宙中饮食体验的视频透视食物覆盖系统
DOI：
10.1145/3491101.3519779
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Kizashi Nakano,Daichi Horita,Naoya Isoyama,Hideaki Uchiyama,Kiyoshi Kiyokawa
中科院分区：
文献类型：
--
作者：
Yamakawa Koichiro;Ishibashi Atsuki;Namiyoshi Toshinobu;Azuma Yuichi;Arakawa Ichiro;Kizashi Nakano,Daichi Horita,Naoya Isoyama,Hideaki Uchiyama,Kiyoshi Kiyokawa
头盔显示器(HMD)的广泛使用允许普通用户每天在社交虚拟环境(VE)或虚拟世界中与他们的朋友互动。然而，戴着头盔在虚拟世界里吃饭并不容易，因为真实环境(RE)是不可见的。目前，用户通过用户面部和头盔之间的缝隙(无)或在VE上叠加视频透视(VST)图像来观看RE的食物，但这些方法会降低临场感。为了允许在VE中自然食用，我们建议使用Ukemochi来改善就餐的存在和便利性。Ukemochi将深度神经网络推断出的食物分割图像无缝地覆盖在VE上。Ukemochi可以同时用作使用OpenVR API创建的VE，并且可以轻松地部署到虚拟世界中。在这项研究中，我们通过比较三种视觉呈现方式(无、VST和Ukemochi)和两种用餐条件(手条件和盘子条件)来评估Ukemochi的效果。实验结果表明，Ukemochi使用户能够保持在VE中的高存在状态，并提高进食便利性。我们相信，我们的研究将为用户提供在虚拟世界吃东西的体验，并鼓励对在虚拟世界吃东西的进一步研究。
The widespread use of Head-Mounted Displays (HMDs) allows ordinary users to interact with their friends daily in social Virtual Environments (VEs) or metaverse. However, it is not easy to eat in a metaverse while wearing an HMD because the Real Environment (RE) is not visible. Currently, users watch the RE’s food through the gap between the user’s face and the HMD (None) or superimposing a video see-through (VST) image on the VE, but these methods reduce the sense of presence. To allow natural eating in a VE, we propose Ukemochi that improves the presence and ease of eating. Ukemochi seamlessly overlays a food segmentation image inferred by deep neural networks on a VE. Ukemochi can be used simultaneously as a VE created with the OpenVR API and can be easily deployed for the metaverse. In this study, we evaluated the effectiveness of Ukemochi by comparing three visual presentation methods (None, VST, and Ukemochi) and two meal conditions (Hand condition and Plate condition). The experimental results demonstrated that Ukemochi enables users to maintain a high presence in VE and improve the ease of eating. We believe that our study will provide users with the experience of eating in the metaverse and encourage further research on eating in the metaverse.