Emergent Reality: Knowledge Formation from Multimodal Learning through Human-Robot Interaction in Extended Reality

涌现现实：扩展现实中通过人机交互进行多模态学习的知识形成

• 批准号: 22K17981
• 负责人: ElHafi Lotfi
• 金额: $ 2.75万
• 依托单位: Ritsumeikan University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Early-Career Scientists
• 财政年份: 2022
• 资助国家: 日本
• 起止时间: 2022-04-01 至 2025-03-31
• 项目状态: 未结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-22K17981/
• 关键词: Extended Reality; Human-Robot Interaction; Multimodal Learning; emergent reality; knowledge formation; multimodal learning; human-robot interaction; extended reality

A human-robot interface using mixed reality (MR) was developed to reduce the user's burden during the interactive, multimodal, and on-site teaching of new knowledge to service robots. The effectiveness of the interface was evaluated using the System Usability Scale (SUS) and NASA Task Load Index (NASA-TLX) in three experimental scenarios: 1) no sharing of inference results with the user, 2) sharing inference results through voice dialogue (baseline), and 3) sharing inference results using the MR interface (proposed). The MR interface significantly reduced temporal, physical, and mental burdens compared to voice dialogue with the robot. The results were presented at IEEE/RSJ IROS 2022 and published in the Nikkan Kogyo Shimbun newspaper.

为了减轻用户在交互式、多模式和现场向服务机器人教授新知识过程中的负担，开发了一种使用混合现实（MR）的人机界面。使用系统可用性量表（SUS）和NASA任务负荷指数（NASA-TLX）在三个实验场景中评估界面的有效性：1）不与用户共享推理结果，2）通过语音对话（基线）共享推理结果，3）使用MR界面（建议）共享推理结果。与与机器人的语音对话相比，MR界面显着减少了时间，身体和精神负担。研究结果在IEEE/RSJ IROS 2022上发表，并发表在《日刊工业新闻》上。

项目成果

Multimodal Object Categorization with Reduced User Load through Human-Robot Interaction in Mixed Reality

通过混合现实中的人机交互进行多模式对象分类，减少用户负载

• DOI: 10.1109/iros47612.2022.9981374
• 发表时间: 2022
• 期刊: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Nakamura Hitoshi;Hafi Lotfi El;Taniguchi Akira;Hagiwara Yoshinobu;Taniguchi Tadahiro
• 通讯作者: Taniguchi Tadahiro