CRII: CHS: Mutual Natural Tangible Interactions with Virtual Characters in Virtual Reality

近年来，虚拟现实（VR）系统越来越多地用于教育、培训和康复等领域。VR系统的有效性依赖于几个关键因素，包括交互。虽然现实生活中的交互包括两个或多个人通过共享有形物体的相互相遇，但迄今为止，大多数VR交互都不是这样的。该项目的目标是建立一个研究计划，最终将缩小真实的和虚拟世界之间的边界，通过从虚拟世界延伸到现实世界的共享对象，将人类用户和虚拟角色之间的相互有形的体现互动。这将增加系统的无缝性和交互的保真度，从而防止技术成为障碍或分心，并使用户能够从VR体验中获得更多好处。研究目标如下：（1）开发一个定制的虚拟现实系统，提供所提出的新的人类用户和虚拟角色之间的相互有形的体现交互。(2)通过控制用户研究和对收集的数据进行统计分析，调查拟议的交互形式对VR中用户表现和体验的影响。(3)通过传播研究结果和思考来影响未来的VR研究。 在设想的定制VR系统中，共享对象将以两种互补的形式表示：物理和虚拟。将共享对象从虚拟世界扩展到真实的世界将通过定制投影表面和定制开发的机械系统来实现。投影表面将被修改，使得只有物理对象的一部分将始终对人类用户可见，而剩余的互补部分将被数字化渲染。机械系统将包括物理对象部件、3D打印附件、微处理器、致动器和运动传感器。这些机械系统将用于在真实的时间中同步物理对象和它们的虚拟对应物的运动;因为代理之一（人或虚拟）影响共享对象的对应形式（物理或虚拟）（例如，通过移动或旋转它），该效果将在另一形式上立即被观察到。 项目成果不仅将使VR系统的用户受益，通过一种新的交互形式增强体验，而且还将通过人机交互的新发现使研究人员受益。研究将分三个阶段进行。第一阶段包括对所提出的相互有形体现交互的调查和设计改进，包括对象和转换模态、自定义硬件和软件、任务和性能测量。在第2阶段，迭代开发和内部测试活动将完成。最后，在第三阶段，将进行包括受控用户研究在内的评估活动，随后将进行数据分析、反思和结果传播。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。

Virtual reality (VR) systems have been increasingly used in recent years in domains such as education, training, and rehabilitation. The effectiveness of VR systems relies on several critical factors, including interaction. Although interaction in real-life includes mutual encounters between two or more individuals through shared tangible objects, most VR interaction to date have not been like that. The goal of this project is to establish a research program that will ultimately shrink the boundary between the real and virtual worlds by incorporating mutual tangible embodied interactions between human users and virtual characters through shared objects that extend from virtual worlds into the real-world. This will increase the seamlessness of the system and the fidelity of the interaction, thereby preventing the technology from being an obstacle or distraction and enabling the user to derive more benefit from the VR experience. The research objectives are as follows: (1) To develop a custom VR system that affords the proposed novel mutual tangible embodied interaction between a human user and a virtual character. (2) To investigate the effects of the proposed form of interaction on user performance and experience in VR through controlled user studies and statistical analyses on the collected data. (3) To impact future VR research by disseminating the findings and reflections. In the envisaged custom VR system, the shared objects will be represented in two complementary forms: physical and virtual. The extension of the shared objects from the virtual world into the real world will be achieved through a custom projection surface and custom-developed mechanical systems. The projection surface will be modified so that only a portion of the physical object will be visible to the human user at all times, while the remaining complementary portion will be rendered digitally. The mechanical systems will include physical object parts, 3D printed attachments, microprocessors, actuators, and motion sensors. These mechanical systems will be employed for synchronizing the movements of the physical objects and their virtual counterparts in real time; as one of the agents (human or virtual) affects the corresponding form (physical or virtual) of the shared object (e.g., by moving or rotating it), the effect will be immediately observable on the other form. Project outcomes will benefit not only the users of VR systems through enhanced experiences with a novel form of interaction but also the researchers through new findings in human-computer interaction. The research will be carried out in three phases. Phase 1 includes the investigation and design refinement of the proposed mutual tangible embodied interaction, including object and transform modality, custom hardware and software, tasks, and performance measures. In Phase 2, iterative development and in-house testing activities will be completed. Finally, in Phase 3, evaluation activities that include controlled user studies will take place, which will be followed by data analysis, reflection, and dissemination of the results.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.