Swarm Robotic Encountered-Type Haptic Interaction in Immersive Environments

我的团队的研究将建立一种新颖的方法，为集群机器人的虚拟和增强现实(VR/AR)提供“碰碰型”触觉。“碰碰型触觉”界面是一种通过重新配置物理环境来为VR/AR提供触觉感觉的新兴方法。与目前的手持或可穿戴触觉设备相比，碰碰型触觉可以物理地渲染用户周围的虚拟表面和对象，这使得VR/AR中的大规模交互能够实现逼真的物理感觉和反馈。此前，还开发了用于碰撞型触觉的其他技术：从接地的机械臂到变形显示器。然而，这些技术经常在其可部署性、可伸缩性和通用性方面带来挑战。我提出的研究将建立一类新的由集群机器人实现的碰碰型触觉交互。在这种新颖的方法中，一群机器人可以动态地构建和重新配置物理对象和空间来模拟虚拟对象，用户可以在VR/AR中与虚拟对象进行物理交互。这种方法将使用户能够在虚拟空间中使用整个身体进行触摸和互动-允许用户在VR/AR环境中行走、坐在上面、抓住、操纵或靠在物体上，就像在物理世界中与它们互动一样。这一新方法承诺为沉浸式环境提供可扩展、可部署和通用的碰碰型触觉。我们将通过三个研究主题来实现研究计划的目标：1)桌面尺度遭遇--变形群体机器人的类型触觉。2)房间--遭遇规模--群体机器人驱动的类型触觉。3)手持式碰碰型触觉，具有可抓取和可变形的物体，由群体机器人组装而成。这些主题中的每一个都旨在支持不同的触觉交互和模式，这将涵盖VR/AR中的广泛交互和应用。该研究项目的成果将在未来的许多应用领域对行业和社会产生重大影响，这些领域包括：手术培训、飞行模拟、工业设计、娱乐、网真和沉浸式环境中的远程协作。

My team's research will establish novel ways of providing "encountered-type" haptics for virtual and augmented reality (VR/AR) with swarm robots. "Encountered-type haptic" interfaces are an emerging approach to providing haptic sensations for VR/AR by reconfiguring physical environments. In contrast to current handheld or wearable haptic devices, encountered-type haptics can physically render virtual surfaces and objects around the user, which enables realistic physical sensations and feedback for large-scale interactions in VR/AR. Previously, other technologies have been developed for encountered-type haptics: from grounded robotic arms to shape-changing displays. However, these technologies often introduce challenges in their deployability, scalability, and generalizability. My proposed research will establish a new class of encountered-type haptic interaction enabled by swarm robots. In this novel approach, a swarm of robots can dynamically construct and reconfigure physical objects and spaces to simulate virtual objects, with which the user can physically interact in VR/AR. This approach will enable users to touch and interact in the virtual space using their whole bodies-allowing the users to walk, sit on, grasp, manipulate, or lean against objects in VR/AR environments, as if interacting with them in the physical world. This new approach promises scalable, deployable, and general--purpose encountered-type haptics for immersive environments. We will achieve the research program's goal through three research themes: 1) Tabletop-scale encountered--type haptics with shape--changing swarm robots. 2) Room--scale encountered--type haptics with swarm robotic actuation. 3) Handheld-scale encountered-type haptics with graspable and transformable objects made by swarm robotic assembly. Each of these themes aims to support different haptic interactions and modalities which will cover a wide range of interactions and applications in VR/AR. The outcomes of this research program will have a substantial impact on industry and society in the future in many application domains that include: surgery training, flight simulation, industrial design, entertainment, telepresence, and remote collaboration in immersive environments.