CHS: Small: Direct physical grasping, manipulation, and tooling of simulated objects

这个项目将开发和评估一种对模拟对象进行物理把握、操作和应用工具的方法，以及一种探索三维信息的方法。该项目基于工具使用的概念，其中手持工具对象用于修改目标对象的属性或外观。该项目整合了人机交互和可视化方面的广泛发现，从双手和有形的用户界面到增强现实。这项研究的核心是一种桌面增强现实系统，它具有立体或单视显示器、触觉定点设备和聚焦在用户手中的摄像头。一方面，用户握住包含虚拟对象的物理线框立方体，另一方面，定点设备的尖端被视觉增强以示出其功能，其将涉及若干可能的工具之一，包括：(A)用于指向、选择和移动对象的探测器，(B)用于过滤、记录和阐述信息的放大或语义镜，以及(C)显示切片或投影视图的切割平面。在显示器上，用户可以通过工具在模拟对象上观看他们的操作的即时、直接的效果。该系统将通过流畅和自然的交互技术支持可视化，在一定程度上提高用户探索和理解3D对象的能力，就像他们手中拿着对象一样。该项目将通过产生跨多个学科的理论和实践进步来提供社会效益，例如更深入地理解心理学理论和计算机算法，这是给人一种无缝印象所必需的，因为他或她正在操纵一个看起来像是在人面前的物体，但实际上它只在那里。将这项工作的实际成果转化为低成本的硬件组件，如头盔3D显示器，应该是相对简单的，这样该项目最终可以为日常用户提供一种直接创建和修改他们心目中的3D打印对象的方法。大学生将帮助开发和评估该系统，从而使这些学生接触到一种新的、潜在的变革性方法，用物理工具扩大模拟空间，从而为更多的学生提供令人兴奋的动手计算机科学学习的机会。骨科医生已经被招募来协助探索使用该系统进行术前手术规划，例如允许探索复杂的3D骨骼结构。开发和评估这种动手增强现实系统中的新交互技术将导致更好地理解如何通过增强工具的使用来提高数据探索的性能。这项研究将调查：用户行为和系统效应的空间搭配；手、物体和环境之间的物理约束；以及本体感觉的更大作用。实验结果将给出跨越人机交互和可视化研究相对不同领域之间的边界的问题的洞察，特别是在双手任务中本体感觉的潜在好处，机械约束和稳定可以在多大程度上改善精确交互任务的性能，以及这些因素如何补偿与在立体或单视显示器上呈现3D信息相关的视觉误差。这项研究将超越实验室，延伸到一个真实而重要的医学领域，这将有助于在实践中验证我们的工作。

This project will develop and evaluate an approach to physical grasping, manipulating, and applying tools to simulated objects, and a means of exploring of three-dimensional information. The project is founded on the concept of tool use, in which handheld tool objects are used to modify the properties or appearance of target objects. The project integrates a wide range of findings in human-computer interaction and visualization, from bimanual and tangible user interfaces to augmented reality. At the core of this research endeavor is a desktop augmented reality system with a stereoscopic or monoscopic display, a haptic pointing device, and a camera focused on the user's hands. In one hand the user holds a physical wireframe cube that contains virtual objects, and in the other hand the pointing device, its tip visually augmented to show its function, which will relate to one of several possible tools, including: (a) a probe for pointing at, selecting, and moving objects, (b) a magnifying or semantic lens for filtering, recoding, and elaborating information, and (c) a cutting plane that shows slices or projection views. On the display, users watch the immediate, direct effects of their actions with the tools on the simulated object. The system will support visualization with fluid and natural interaction techniques, improving the ability of users to explore and understand 3D objects to some extent as if they were holding the objects in their hands. The project will provide societal benefits by generating theoretical and practical advances across multiple disciplines, such as a deeper understanding of the psychological theory and computer algorithms that are needed to give a person a seamless impression that he or she is manipulating an object that appears to be physically in front of the person, but which is only there virtually. It should be relatively straightforward to transition the practical outcome of this work to low-cost hardware components such as head-mounted 3D displays, such that the project could ultimately provide everyday users with a means of directly creating and modifying objects that they have in mind for 3D printing. University students will help to develop and evaluate the system, thus exposing these students to a new and potentially transformative approach to augmenting a simulated space with physical tools, and thus providing a larger population of students with opportunities for exciting hands-on computer science learning. Orthopedic surgeons have been recruited to assist in exploring the use of the system for preoperative surgical planning such as by permitting the exploration of complex 3D bone structures.Development and evaluation of the novel interaction techniques in this hands-on augmented reality system will lead to a better understanding of how performance may be improved in data exploration with augmented tool use. The research will investigate: the spatial collocation of user actions and system effects; physical constraints between the hands, objects, and the environment; and a greater role for proprioception. Experimental results will give insight into questions that cross the boundaries between relatively disparate areas of research in HCI and visualization, specifically the potential benefits of proprioception in bimanual tasks, the extent to which mechanical constraints and stabilization can improve performance in precise interaction tasks, and how these factors may compensate for the visual errors associated with presenting 3D information on a stereo or monoscopic display. The research will extend beyond the laboratory to a real and important medical domain, which will help validate our work in practice.