STTR Phase I: Virtual Interfaces using Multi-Protocol, Augmented-Reality Activation-Based Control Transfer

此小型企业技术转让(STTR)第一阶段项目的更广泛影响/商业潜力提供了可动态定制的虚拟界面，以应对用户在增强现实(AR)产品中采用的挑战。该项目克服了AR以前的限制，例如缺乏互操作性、用户接受度、可扩展性和应用程序感知的用户意图识别。虽然AR市场预计将会增长，但可穿戴式协作仍有未开发的潜力。由该项目授权的用户可能能够轻松地与熟练的专家一起协作，以及在同一媒介上使用机器人界面。通过情景感知的动态控制，制造企业可以降低人为错误的风险，同时提高履行能力，因为在家工作的员工可以保持物理联系，保持最小限度的投入。该项目中提出的增强现实技术创新应用于多个行业，包括制造、培训和应急模拟。该小型企业技术转移(STTR)第一阶段项目旨在为虚拟界面的可扩展实验创建一个框架，使用户能够以安全和隐私的方式使用手势交互来搜索、标记和存储有意义的数据。该项目旨在提供与无线设备的基于环境的免提交互，同时最大限度地减少在受控环境中设置传感器(如光学组件或麦克风)的需要，并为最终用户提供视觉反馈和可访问性。手腕佩戴的可穿戴设备捕获预先训练的手势数据，并且一副增强现实(AR)眼镜识别和分析所述手势的上下文中的对象，以控制诸如机器人的机电设备来执行职业任务。这个动态过程使用了用户的凝视区域、物理位置和计算机应用等因素。手势控制的改进可能允许单独的视觉单元，如AR眼镜，无线接收和显示手势意图，并执行上下文感知的、有意义的交互，在与特定使用案例相关联的多个设备或数字标记对象上有明确的区分。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this Small Business Technology Transfer (STTR) Phase I project provides virtual interfaces that can be dynamically customized to address the challenge of user adoption in augmented reality (AR) products. This project overcomes previous limitations for AR such as the lack of interoperability, user-acceptance, scalability, and application-aware user intent recognition. While the AR market is projected to grow, wearable collaboration has unexplored potential. Users empowered by this project may be able to easily collaborate alongside skilled experts as well as robotic interfaces in the same medium. Through contextually-aware dynamic controls, manufacturing businesses can lower the risk of human-error while increasing fulfillment capabilities, as employees working from home can stay physically connected and remain engaged with minimal compromise. The AR innovation proposed in this project has application across multiple industries, including manufacturing, training, and emergency simulations.This Small Business Technology Transfer (STTR) Phase I project aims to create a framework for scalable experimentation of virtual interfaces that empower users to search, tag, and store meaningful data using gesture interactions in a secure and private manner. This project seeks to provide context-based, hands-free interaction with wireless devices while minimizing the need to set up sensors (such as optical components or microphones) in controlled environments and providing visual feedback and accessibility to the end user. A wrist-worn wearable captures pre-trained gesture data, and a pair of augmented reality (AR) glasses identifies and analyzes an object within the context of said gestures to control a mechatronic device, such as a robot, to perform occupational tasks. This dynamic process uses factors such as the user’s gaze area, physical location, and computer application. The improvement upon gesture control may allow a separate visual unit, such as the AR glasses, to wirelessly receive and display the gesture intents and perform contextually-aware, meaningful interactions with a clear distinction across a plurality of devices or digitally tagged objects associated with a specific use-case.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.