II-New: Flexible User Interaction Instrumentation for Ubiquitous and Immersive Computing Environments

II-新：适用于无处不在的沉浸式计算环境的灵活的用户交互工具

• 批准号: 1730033
• 负责人: Andrea Kleinsmith
• 金额: $ 35.67万
• 依托单位: University of Maryland Baltimore County
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-07-01 至 2020-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1730033&HistoricalAwards=false
• 关键词: II; New; Flexible; User; Interaction

Smart phones, networked devices, and commercially available augmented and virtual reality kits are making ubiquitous and immersive interactive systems increasingly commonplace. Designing these systems, and the applications that use them, requires the ability to study how system features and human behavior affect each other in natural contexts. This award will provide an interdisciplinary research team with equipment for capturing both individual and small group behavioral data in situ, including body motion, eye gaze, mental workload, and emotion. The infrastructure will support projects at the team's institution in a number of domains, including autonomous vehicle use by visually impaired users, augmented reality training for emergency medical responders, and collaborative scientific discovery in virtual visualization environments. A doctoral student with related research interests will coordinate management of and training on the infrastructure, developing both technical and research skills. The team will also use the equipment to provide enhanced research opportunities for undergraduates from a number of disciplines, institutions, and backgrounds. Much of the planned infrastructure uses next-generation versions of tools that the team already has expertise with. This reduces deployment risks and allows them to augment existing projects with new capabilities while enabling new directions for research. In most cases, this is a transformation from fixed lab-based sensing to unconstrained, mobile data collection in the field. The new capabilities include five main data sources. One is body motion data, which will be collected using an industry standard infrared-based motion capture system that can flexibly capture the movements of individuals or dyads. A second is location and gait capture, for both individuals and groups, through an unobtrusive, configurable system of floor-mounted force plates. A third is eye gaze data, collected through a portable headset that captures eye fixations and pupil dilation to support monitoring visual attention. A fourth is electroencephalogram (EEG) data, collected through a portable dry sensor-based headset, that can monitor workload, affect, and facial features as well as support prototyping of brain-computer interfaces (BCIs). A fifth is physiological data including temperature, pulse, arm motion, and arousal, collected through a inconspicuous wristband that includes photoplethysmography, accelerometer, and electrodermal sensors. Each individual data stream comes with accompanying analytic software; collectively, data will be managed through a commercially available tool for analyzing events synchronized across multiple parallel data streams. Key innovations of this award this award are its novel data integration strategies and cross-disciplinary application areas.

智能手机、联网设备以及商业上可用的增强和虚拟现实工具包正在使无处不在、身临其境的交互系统变得越来越常见。设计这些系统和使用它们的应用程序，需要研究系统特征和人类行为在自然环境中如何相互影响的能力。该奖项将为一个跨学科的研究团队提供设备，以在现场捕获个人和小群体行为数据，包括身体运动、眼睛凝视、精神负荷和情绪。该基础设施将支持该团队机构在多个领域的项目，包括视障用户自动驾驶车辆的使用，紧急医疗响应人员的增强现实培训，以及虚拟可视化环境中的协作科学发现。具有相关研究兴趣的博士生将协调基础设施的管理和培训，发展技术和研究技能。该团队还将使用这些设备为来自多个学科、机构和背景的本科生提供更多的研究机会。计划中的大部分基础设施使用团队已经熟练掌握的下一代工具版本。这降低了部署风险，并允许他们使用新功能增强现有项目，同时支持新的研究方向。在大多数情况下，这是一种从基于实验室的固定传感到现场无约束移动数据收集的转变。新功能包括五个主要数据源。一种是身体运动数据，将使用行业标准的基于红外的运动捕捉系统收集这些数据，该系统可以灵活地捕捉个体或二人组的运动。第二个是位置和步态捕捉，无论是个人还是群体，通过一个不显眼的、可配置的落地式测力板系统。第三个是眼睛凝视数据，通过便携式耳机收集，捕捉眼睛的凝视和瞳孔放大，以支持监控视觉注意力。第四种是脑电(EEG)数据，通过便携式干式传感器耳机收集，可以监控工作量、影响和面部特征，并支持脑机接口(BCI)的原型设计。第五种是生理数据，包括体温、脉搏、手臂运动和觉醒，通过一个不显眼的腕带收集，其中包括光体积描记仪、加速计和电皮肤传感器。每个单独的数据流都附带分析软件；总的来说，数据将通过商业上可用的工具进行管理，该工具用于分析跨多个并行数据流同步的事件。该奖项的关键创新之处在于其新颖的数据集成策略和跨学科的应用领域。

项目成果

Virtual pointer for gaze guidance in laparoscopic surgery

• DOI: 10.1007/s00464-019-07141-x
• 发表时间: 2019-10-04
• 期刊: SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES
• 影响因子: 3.1
• 作者: Feng, Yuanyuan;McGowan, Hannah;Mentis, Helena
• 通讯作者: Mentis, Helena

Distinguishing Anxiety Subtypes of English Language Learners Towards Augmented Emotional Clarity

• DOI: 10.1007/978-3-030-52240-7_29
• 发表时间: 2020-06-10
• 期刊: Artificial Intelligence in Education
• 作者: Lee H;Mandalapu V;Kleinsmith A;Gong J
• 通讯作者: Gong J

How Trainees Use the Information from Telepointers in Remote Instruction

学员如何在远程教学中使用远程指示器的信息

• DOI: 10.1145/3359195
• 发表时间: 2019
• 期刊: Proceedings of the ACM on Human-Computer Interaction
• 作者: Semsar, Azin;McGowan, Hannah;Feng, Yuanyuan;Zahiri, H. Reza;Park, Adrian;Kleinsmith, Andrea;Mentis, Helena
• 通讯作者: Mentis, Helena

Communication Cost of Single-user Gesturing Tool in Laparoscopic Surgical Training

腹腔镜手术培训中单用户手势工具的通信成本

• DOI: 10.1145/3290605.3300841
• 发表时间: 2019
• 期刊: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
• 作者: Feng, Yuanyuan;Mentis, Helena M.;Li, Katie;Semsar, Azin;McGowan, Hannah;Mun, Jacqueline;Zahiri, H. Reza;George, Ivan;Park, Adrian;Kleinsmith, Andrea
• 通讯作者: Kleinsmith, Andrea

A virtual pointer to support the adoption of professional vision in laparoscopic training

• DOI: 10.1007/s11548-018-1792-9
• 发表时间: 2018-09-01
• 期刊: INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY
• 影响因子: 3
• 作者: Feng, Yuanyuan;McGowan, Hannah;Mentis, Helena M.
• 通讯作者: Mentis, Helena M.