II-New: Collaborative: A Mixed Reality Environment for Enabling Everywhere Data-Centric Work

这个基础设施项目将开发一个名为OpenMR的开源软件工具包，以支持构建“混合现实”数据分析系统，该系统使用微软Hololens和Oculus Rift等新型显示设备将数据投射到物理世界中。通过OpenMR，这些轻量级的、可穿戴的移动设备将接入云托管的数据密集型基础设施，其目标是开发允许用户在任何地方执行数据密集型任务的系统，而不需要由专用本地计算机支持的重型专用大格式显示器。为了进行这项研究，研究人员将获得专用云计算服务器（以支持数据分析）和混合现实硬件设备（以创建接口）。他们将开发OpenMR来连接这些硬件，以支持常见的分析任务，如选择、过滤和分类数据，并在物理世界中创建数据显示。为了展示该工具包并推进数据分析研究，他们将为需要分析天气、生物学和医学成像等领域大量数据的研究人员构建许多混合现实原型界面。除了推进这些特定的研究领域，与真实用户一起研究这些原型将支持围绕底层数据分析技术的研究，人们如何与物理世界中的数据交互的认知科学，以及构建混合现实系统所需的设计原则。反过来，这将使这些新兴技术更有可能成功和传播，并增加为这些系统找到潜在“杀手级应用”的机会。该基础设施还将直接支持合作大学在数据可视化、计算机图形学、计算机视觉和机器学习方面的教育和研究，而工具包的发布将使更广泛的社区受益。这项研究是及时和重要的，因为随着智能设备，特别是虚拟和混合现实设备，如谷歌眼镜，微软Hololens， Oculus Rift和谷歌Cardboard，变得司空见惯，这些设备在与数字信息交互时，相对于传统的笔记本电脑和数字计算机将发挥越来越重要的作用。该项目的长期愿景是开发一个混合现实研究基础设施，以支持无处不在的以数据为中心的创新，提供身临其境、直观、无位置限制、先进的机器学习、数据分析、简化、汇总和存储工具。这包括通过OpenMR开源工具包对混合现实空间中以数据为中心的工作的完整管道的高级支持，包括前端可视化和交互，利用可用渲染空间和硬件的意识以及2D和3D空间中的有效可视化模式来优化交互；中间层数据分析和机器学习的关键组成部分包括自动、通用特征工程和分类性能的联合优化以及鉴别特征的有效识别；以及服务器上的高性能计算和成本敏感型作业管理。该团队将通过多种机制评估OpenMR的效率、稳定性、可扩展性、功能、灵活性和易用性，包括设计过程的自我评估和文档，领域专家的审查，以及专家和新手用户对上述特定应用领域的数据分析任务的评估。该工具包本身将在项目的第三年期间在GitHub开源平台上发布，届时它将达到初始的成熟度和实用性水平。调查人员将通过Youtube频道发布OpenMR，并提供一系列演示视频；拓展对沉浸式可视化、视觉分析、多感官人机交互、人在环机器学习和高性能计算感兴趣的相关研究人员；以及与学生、技术、学术、研究和服务计算军团联盟的本科生合作。

This infrastructure project will develop an open source software toolkit, called OpenMR, to support building "mixed reality" data analysis systems that project data into the physical world using a new class of display devices such as Microsoft Hololens and Oculus Rift. Through OpenMR, these lightweight, wearable, mobile devices will tap into data-intensive infrastructures hosted in the cloud, with the goal of developing systems that allow users to perform data-intensive tasks from anywhere, without requiring heavy dedicated large-format displays supported by dedicated local computers. To pursue this research, the investigators will acquire both dedicated cloud-computing servers (to support data analysis) and mixed reality hardware devices (to create the interfaces). They will develop OpenMR to connect this hardware, to support common analysis tasks such as selecting, filtering, and classifying data, and to create data displays in the physical world. To both demonstrate the toolkit and advance data analysis research, they will build a number of prototype mixed reality interfaces for researchers whose work requires analyzing a large amount of data in domains including weather, biology, and medical imaging. In addition to advancing those specific research areas, studying these prototypes with real users will support research around the underlying data analysis techniques, the cognitive science of how people interact with data in the physical world, and the design principles needed to build mixed reality systems. This, in turn, will make these emerging technologies more likely to succeed and spread, and increase the chance of finding potential 'killer apps' for these systems. The infrastructure will also directly support education and research at the partner universities around data visualization, computer graphics, computer vision, and machine learning, while the release of the toolkit will benefit the wider community. This research is timely and important because as smart devices, in particular virtual and mixed reality devices such as Google Glass, Microsoft Hololens, Oculus Rift and Google Cardboard, become commonplace, these devices will play an increasingly important role relative to traditional laptop and digital computers when interacting with digital information. The long-term vision of the project is to develop a mixed reality research infrastructure to support everywhere data-centric innovations, providing immersive, intuitive, location-free, advanced machine learning, data analysis, reduction, summary and storage tools. This includes advanced support for the full pipeline of data-centric work in mixed reality spaces through the OpenMR open source toolkit, including front end visualization and interaction that leverages awareness of available rendering spaces and hardware along with effective visualization patterns in 2D and 3D spaces to optimize interaction; key components of data analysis and machine learning on the middle layers including automatic, generic feature engineering and joint optimization of classification performance and effective identification of discriminating features; and high-performance computing and cost-sensitive job management on the server. The team will evaluate OpenMR's efficiency, stability, scalability, functionality, flexibility, and ease of adoption through a number of mechanisms, including self-evaluations and documentation of the design process, review from domain experts, and evaluation with both expert and novice users on data analysis tasks that cur across the specific application domains described above. The toolkit itself will be released on the GitHub open source platform during the third year of the project after it has reached an initial level of maturity and usefulness. The investigators will publicize OpenMR through a Youtube channel with a set of demonstration videos; outreach to relevant researchers interested in immersive visualization, visual analytics, multi-sensory human-computer interaction, machine learning with human-in-the-loop, and high-performance computing; and collaboration with undergraduates in the Students, Technology, Academia, Research, and Service Computing Corps consortium.