II-New: Flexible User Interaction Instrumentation for Ubiquitous and Immersive Computing Environments

智能手机、联网设备以及商用增强现实和虚拟现实套件正在使无处不在的沉浸式交互系统变得越来越普遍。设计这些系统，以及使用它们的应用程序，需要研究系统特征和人类行为在自然环境中如何相互影响的能力。该奖项将为一个跨学科研究团队提供设备，用于捕获个人和小组行为数据，包括身体运动、眼睛注视、精神负荷和情绪。该基础设施将支持该团队所在机构在多个领域的项目，包括视障用户使用的自动驾驶汽车、紧急医疗响应人员的增强现实培训，以及虚拟可视化环境中的协作科学发现。具有相关研究兴趣的博士生将协调基础设施的管理和培训，发展技术和研究技能。该团队还将利用这些设备为来自不同学科、机构和背景的本科生提供更多的研究机会。许多计划中的基础设施使用了团队已经掌握的下一代工具版本。这降低了部署风险，并允许他们在为研究提供新方向的同时，用新功能增强现有项目。在大多数情况下，这是从基于实验室的固定传感到现场不受约束的移动数据收集的转变。新功能包括五个主要数据源。一个是身体运动数据，将使用工业标准的基于红外的运动捕捉系统收集，该系统可以灵活地捕捉个人或二人组的运动。第二个是位置和步态捕获，个人和群体，通过一个不显眼的，可配置的系统安装在地板上的力板。第三个是眼睛注视数据，通过一个便携式耳机收集，捕捉眼睛的注视和瞳孔扩张，以支持监控视觉注意力。第四个是脑电图（EEG）数据，通过基于干燥传感器的便携式耳机收集，可以监测工作负荷、情绪和面部特征，并支持脑机接口（bci）的原型设计。第五种是生理数据，包括温度、脉搏、手臂运动和觉醒，这些数据是通过一个不显眼的腕带收集的，腕带包括光电体积脉搏仪、加速度计和皮肤电传感器。每个单独的数据流都附带分析软件；总的来说，数据将通过商业上可用的工具进行管理，该工具用于分析跨多个并行数据流同步的事件。该奖项的主要创新是其新颖的数据集成策略和跨学科应用领域。

Smart phones, networked devices, and commercially available augmented and virtual reality kits are making ubiquitous and immersive interactive systems increasingly commonplace. Designing these systems, and the applications that use them, requires the ability to study how system features and human behavior affect each other in natural contexts. This award will provide an interdisciplinary research team with equipment for capturing both individual and small group behavioral data in situ, including body motion, eye gaze, mental workload, and emotion. The infrastructure will support projects at the team's institution in a number of domains, including autonomous vehicle use by visually impaired users, augmented reality training for emergency medical responders, and collaborative scientific discovery in virtual visualization environments. A doctoral student with related research interests will coordinate management of and training on the infrastructure, developing both technical and research skills. The team will also use the equipment to provide enhanced research opportunities for undergraduates from a number of disciplines, institutions, and backgrounds. Much of the planned infrastructure uses next-generation versions of tools that the team already has expertise with. This reduces deployment risks and allows them to augment existing projects with new capabilities while enabling new directions for research. In most cases, this is a transformation from fixed lab-based sensing to unconstrained, mobile data collection in the field. The new capabilities include five main data sources. One is body motion data, which will be collected using an industry standard infrared-based motion capture system that can flexibly capture the movements of individuals or dyads. A second is location and gait capture, for both individuals and groups, through an unobtrusive, configurable system of floor-mounted force plates. A third is eye gaze data, collected through a portable headset that captures eye fixations and pupil dilation to support monitoring visual attention. A fourth is electroencephalogram (EEG) data, collected through a portable dry sensor-based headset, that can monitor workload, affect, and facial features as well as support prototyping of brain-computer interfaces (BCIs). A fifth is physiological data including temperature, pulse, arm motion, and arousal, collected through a inconspicuous wristband that includes photoplethysmography, accelerometer, and electrodermal sensors. Each individual data stream comes with accompanying analytic software; collectively, data will be managed through a commercially available tool for analyzing events synchronized across multiple parallel data streams. Key innovations of this award this award are its novel data integration strategies and cross-disciplinary application areas.