CHS: Small: Collaborative Research: Wearable Fingertip Haptic Devices for Virtual and Augmented Reality: Design, Control, and Predictive Tracking

CHS：小型：协作研究：用于虚拟和增强现实的可穿戴指尖触觉设备：设计、控制和预测跟踪

• 批准号: 1812933
• 负责人: Yu Sun
• 金额: $ 17.33万
• 依托单位: University of South Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1812933&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Wearable

In the real world, people rely heavily on haptic (force and tactile) feedback to manipulate and explore objects. However, in virtual and augmented reality environments, haptic sensations must be artificially generated. Advances in both sensing and feedback are needed to create more realistic virtual scenarios for human interaction, education, and training. This research will implement wearable fingertip haptic feedback systems that enhance human perception and task performance in virtual and augmented reality, and will provide the field of computer-mediated haptics with important new design and rendering insights for a promising form of wearable haptic device. Successful development of fingertip haptic devices and improved tracking methods will lower the cost of integrating haptic technology into virtual reality, and yield more realistic and compelling virtual experiences. Effective haptic feedback systems for virtual reality utilizing the developed technologies will have broad impact by improving human health and well-being through a myriad of applications such as training for critical tasks like surgery and defusing of explosive ordnance, tactile communication to enable design and e-commerce, and immersion in virtual worlds for enhanced education and training. Project outcomes will be disseminated through demonstrations of educational applications, expansion of an online course on haptics, and publicly available software and data. Outreach programs, public lab tours, and mentoring of female and minority graduate students, undergraduates, and high school students will broaden participation of underrepresented groups in engineering.This project has three main technical components. First, instrumented objects will be developed that measure force interactions and integrate that data with external measurements of finger pose. Second, new wearable haptic devices will be developed that use a combination of local kinesthetic feedback and skin deformation to provide realistic haptic feedback with minimal encumbrance. Modular haptic device designs will facilitate rapid testing of various design choices. Then the basic design will be optimized in terms of control strategy, degrees of freedom, skin contact conditions, and finger grounding to best match the real-world grasping and manipulation data collected earlier. Third, predictive tracking algorithms will be developed for grasping based on characteristic behaviors observed in real-world grasping and manipulation data. All of the haptic devices and tracking methods will be evaluated both in virtual and in augmented reality environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在现实世界中，人们严重依赖触觉（力和触觉）反馈来操纵和探索物体。 但是，在虚拟和增强的现实环境中，必须人为地产生触觉感觉。 需要进行感应和反馈的进步，以创建更现实的虚拟场景，用于人类的互动，教育和培训。 这项研究将实施可穿戴的触觉反馈系统，从而在虚拟和增强现实中增强人类的看法和任务绩效，并将为计算机介导的触觉提供重要的新设计，并为可穿戴触觉设备的有前途的形式提供重要的新设计和提供见解。 成功开发指尖触觉设备和改进的跟踪方法将降低将触觉技术整合到虚拟现实中的成本，并产生更现实，更具吸引力的虚拟体验。 利用已发达技术的有效触觉反馈系统将通过通过无数的应用来改善人类健康和福祉，例如诸如手术培训以及爆炸性的触觉沟通，启用设计和电子商务，以及在虚拟的教育和培训中浸入式的爆炸性，将产生广泛的影响。 项目成果将通过展示教育应用，扩展有关触觉的在线课程以及公开可用的软件和数据来传播。 外展计划，公共实验室旅行以及对女性和少数族裔研究生，本科生和高中生的指导将扩大代表性不足的团体在工程中的参与。该项目具有三个主要的技术组成部分。 首先，将开发仪器的对象，以测量力相互作用，并将该数据与手指姿势的外部测量集成在一起。 其次，将开发新的可穿戴触觉设备，结合局部动力学反馈和皮肤变形，以最少的负担提供逼真的触觉反馈。 模块化触觉设备设计将有助于快速测试各种设计选择。 然后，将根据控制策略，自由度，皮肤接触条件和手指接地来优化基本设计，以最佳地匹配现实世界中的抓地力和操纵数据。 第三，将根据在现实世界抓取和操纵数据中观察到的特征行为来开发预测性跟踪算法。 所有触觉设备和跟踪方法都将在虚拟和增强现实环境中进行评估。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的智力优点和更广泛影响的评估评估标准来通过评估来获得支持的。

项目成果

Estimating Motion Codes from Demonstration Videos

• DOI: 10.1109/iros45743.2020.9341065
• 发表时间: 2020-07
• 期刊: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Maxat Alibayev;D. Paulius;Yu Sun

Multi-Object Grasping - Types and Taxonomy

多对象抓取 - 类型和分类

• DOI: 10.1109/icra46639.2022.9812388
• 发表时间: 2022
• 期刊: 2022 International Conference on Robotics and Automation (ICRA
• 作者: Sun, Yu;Amatova, Eliza;Chen, Tianze
• 通讯作者: Chen, Tianze

A Motion Taxonomy for Manipulation Embedding

• DOI: 10.15607/rss.2020.xvi.045
• 发表时间: 2020-07
• 期刊: ArXiv
• 作者: D. Paulius;Nicholas Eales;Yu Sun

Generalizing Learned Manipulation Skills in Practice

在实践中推广所学的操作技能

• DOI: 10.1109/iros45743.2020.9340739
• 发表时间: 2020
• 期刊: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Wilches, Juan;Huang, Yongqiang;Sun, Yu
• 通讯作者: Sun, Yu

Accurate Pouring using Model Predictive Control Enabled by Recurrent Neural Network

• DOI: 10.1109/iros40897.2019.8967802
• 发表时间: 2019-11
• 期刊: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Tianze Chen;Yongqiang Huang;Yu Sun