Swarm Robotic Encountered-Type Haptic Interaction in Immersive Environments

我的团队的研究将建立新的方法，为虚拟和增强现实（VR/AR）机器人提供“遭遇式”触觉。“接触型触觉”界面是一种通过重新配置物理环境来为VR/AR提供触觉感觉的新兴方法。与目前的手持式或可穿戴式触觉设备相比，遭遇式触觉可以物理地呈现用户周围的虚拟表面和物体，从而为VR/AR中的大规模交互提供真实的物理感觉和反馈。在此之前，已经开发了其他技术来实现偶遇型触觉：从接地的机械臂到可变形的显示器。然而，这些技术常常在可部署性、可伸缩性和通用性方面带来挑战。我提出的研究将建立一种由群体机器人实现的新型遭遇型触觉交互。在这种新颖的方法中，一群机器人可以动态地构建和重新配置物理对象和空间来模拟虚拟对象，用户可以在VR/AR中与虚拟对象进行物理交互。这种方法将使用户能够使用他们的整个身体在虚拟空间中触摸和互动-允许用户在VR/AR环境中行走，坐在，抓住，操纵或倚靠物体，就像在物理世界中与它们互动一样。这种新方法有望为沉浸式环境提供可扩展、可部署和通用的遭遇式触觉。我们将通过三个研究主题来实现研究计划的目标：1)桌面尺度的遇到型触觉与形状变化的群体机器人。2)具有群体机器人驱动的房间尺度偶遇型触觉。3)手持规模的遭遇式触觉，通过群体机器人组装制造可抓取和可变形的物体。每个主题都旨在支持不同的触觉交互和模式，这将涵盖VR/AR中的广泛交互和应用。该研究项目的成果将在未来的许多应用领域对工业和社会产生重大影响，包括：手术培训、飞行模拟、工业设计、娱乐、远程呈现和沉浸式环境中的远程协作。

My team's research will establish novel ways of providing "encountered-type" haptics for virtual and augmented reality (VR/AR) with swarm robots. "Encountered-type haptic" interfaces are an emerging approach to providing haptic sensations for VR/AR by reconfiguring physical environments. In contrast to current handheld or wearable haptic devices, encountered-type haptics can physically render virtual surfaces and objects around the user, which enables realistic physical sensations and feedback for large-scale interactions in VR/AR. Previously, other technologies have been developed for encountered-type haptics: from grounded robotic arms to shape-changing displays. However, these technologies often introduce challenges in their deployability, scalability, and generalizability. My proposed research will establish a new class of encountered-type haptic interaction enabled by swarm robots. In this novel approach, a swarm of robots can dynamically construct and reconfigure physical objects and spaces to simulate virtual objects, with which the user can physically interact in VR/AR. This approach will enable users to touch and interact in the virtual space using their whole bodies-allowing the users to walk, sit on, grasp, manipulate, or lean against objects in VR/AR environments, as if interacting with them in the physical world. This new approach promises scalable, deployable, and general--purpose encountered-type haptics for immersive environments. We will achieve the research program's goal through three research themes: 1) Tabletop-scale encountered--type haptics with shape--changing swarm robots. 2) Room--scale encountered--type haptics with swarm robotic actuation. 3) Handheld-scale encountered-type haptics with graspable and transformable objects made by swarm robotic assembly. Each of these themes aims to support different haptic interactions and modalities which will cover a wide range of interactions and applications in VR/AR. The outcomes of this research program will have a substantial impact on industry and society in the future in many application domains that include: surgery training, flight simulation, industrial design, entertainment, telepresence, and remote collaboration in immersive environments.