II-New: Collaborative: A Mixed Reality Environment for Enabling Everywhere Data-Centric Work

II-新：协作：支持无处不在的以数据为中心的工作的混合现实环境

• 批准号: 1629890
• 负责人: Jian Huang
• 金额: $ 35.07万
• 依托单位: University of Tennessee Knoxville
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-10-01 至 2020-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1629890&HistoricalAwards=false
• 关键词: II; New; Collaborative; Mixed; Reality

This infrastructure project will develop an open source software toolkit, called OpenMR, to support building "mixed reality" data analysis systems that project data into the physical world using a new class of display devices such as Microsoft Hololens and Oculus Rift. Through OpenMR, these lightweight, wearable, mobile devices will tap into data-intensive infrastructures hosted in the cloud, with the goal of developing systems that allow users to perform data-intensive tasks from anywhere, without requiring heavy dedicated large-format displays supported by dedicated local computers. To pursue this research, the investigators will acquire both dedicated cloud-computing servers (to support data analysis) and mixed reality hardware devices (to create the interfaces). They will develop OpenMR to connect this hardware, to support common analysis tasks such as selecting, filtering, and classifying data, and to create data displays in the physical world. To both demonstrate the toolkit and advance data analysis research, they will build a number of prototype mixed reality interfaces for researchers whose work requires analyzing a large amount of data in domains including weather, biology, and medical imaging. In addition to advancing those specific research areas, studying these prototypes with real users will support research around the underlying data analysis techniques, the cognitive science of how people interact with data in the physical world, and the design principles needed to build mixed reality systems. This, in turn, will make these emerging technologies more likely to succeed and spread, and increase the chance of finding potential 'killer apps' for these systems. The infrastructure will also directly support education and research at the partner universities around data visualization, computer graphics, computer vision, and machine learning, while the release of the toolkit will benefit the wider community. This research is timely and important because as smart devices, in particular virtual and mixed reality devices such as Google Glass, Microsoft Hololens, Oculus Rift and Google Cardboard, become commonplace, these devices will play an increasingly important role relative to traditional laptop and digital computers when interacting with digital information. The long-term vision of the project is to develop a mixed reality research infrastructure to support everywhere data-centric innovations, providing immersive, intuitive, location-free, advanced machine learning, data analysis, reduction, summary and storage tools. This includes advanced support for the full pipeline of data-centric work in mixed reality spaces through the OpenMR open source toolkit, including front end visualization and interaction that leverages awareness of available rendering spaces and hardware along with effective visualization patterns in 2D and 3D spaces to optimize interaction; key components of data analysis and machine learning on the middle layers including automatic, generic feature engineering and joint optimization of classification performance and effective identification of discriminating features; and high-performance computing and cost-sensitive job management on the server. The team will evaluate OpenMR's efficiency, stability, scalability, functionality, flexibility, and ease of adoption through a number of mechanisms, including self-evaluations and documentation of the design process, review from domain experts, and evaluation with both expert and novice users on data analysis tasks that cur across the specific application domains described above. The toolkit itself will be released on the GitHub open source platform during the third year of the project after it has reached an initial level of maturity and usefulness. The investigators will publicize OpenMR through a Youtube channel with a set of demonstration videos; outreach to relevant researchers interested in immersive visualization, visual analytics, multi-sensory human-computer interaction, machine learning with human-in-the-loop, and high-performance computing; and collaboration with undergraduates in the Students, Technology, Academia, Research, and Service Computing Corps consortium.

该基础设施项目将开发一个名为OpenMR的开源软件工具包，以支持构建“混合现实”数据分析系统，该系统使用Microsoft Hololens和Oculus Rift等新型显示设备将数据投射到物理世界中。通过OpenMR，这些轻量级、可穿戴的移动的设备将利用托管在云中的数据密集型基础设施，目标是开发允许用户从任何地方执行数据密集型任务的系统，而不需要由专用本地计算机支持的沉重的专用大格式显示器。 为了进行这项研究，研究人员将获得专用的云计算服务器（以支持数据分析）和混合现实硬件设备（以创建界面）。 他们将开发OpenMR来连接这些硬件，以支持常见的分析任务，如选择，过滤和分类数据，并在物理世界中创建数据显示。为了展示工具包和推进数据分析研究，他们将为需要分析天气、生物和医学成像等领域大量数据的研究人员构建一些原型混合现实接口。 除了推进这些特定的研究领域，与真实的用户一起研究这些原型将支持围绕底层数据分析技术的研究，人们如何与物理世界中的数据交互的认知科学，以及构建混合现实系统所需的设计原则。 这反过来又将使这些新兴技术更有可能成功和传播，并增加为这些系统找到潜在“杀手级应用程序”的机会。 该基础设施还将直接支持合作大学围绕数据可视化、计算机图形学、计算机视觉和机器学习的教育和研究，而工具包的发布将使更广泛的社区受益。 这项研究是及时和重要的，因为随着智能设备，特别是虚拟和混合现实设备，如谷歌眼镜，微软Hololens，Oculus Rift和谷歌Cardboard，变得越来越普遍，这些设备将在与数字信息交互时发挥越来越重要的作用。该项目的长期愿景是开发一个混合现实研究基础设施，以支持以数据为中心的创新，提供沉浸式，直观，无位置，先进的机器学习，数据分析，简化，摘要和存储工具。 这包括通过OpenMR开源工具包对混合现实空间中以数据为中心的工作的完整管道提供高级支持，包括前端可视化和交互，利用对可用渲染空间和硬件的感知沿着2D和3D空间中的有效可视化模式来优化交互;中间层的数据分析和机器学习的关键组件，包括自动，通用特征工程和分类性能的联合优化以及鉴别特征的有效识别;以及服务器上的高性能计算和成本敏感型作业管理。 该团队将通过多种机制评估OpenMR的效率、稳定性、可扩展性、功能性、灵活性和易于采用性，包括设计过程的自我评估和文档、领域专家的审查以及专家和新手用户对上述特定应用领域的数据分析任务的评估。 该工具包本身将在项目的第三年在GitHub开源平台上发布，届时它将达到成熟度和实用性的初始水平。 研究人员将通过一组演示视频的Youtube频道宣传OpenMR;与对沉浸式可视化，视觉分析，多感官人机交互，人机回路机器学习和高性能计算感兴趣的相关研究人员进行外联;并与学生，技术，学术界，研究和服务计算军团联盟的本科生合作。