CHS: Small: Collaborative Research: Wearable Fingertip Haptic Devices for Virtual and Augmented Reality: Design, Control, and Predictive Tracking

CHS：小型：协作研究：用于虚拟和增强现实的可穿戴指尖触觉设备：设计、控制和预测跟踪

• 批准号: 1812933
• 负责人: Yu Sun
• 金额: $ 17.33万
• 依托单位: University of South Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1812933&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Wearable

In the real world, people rely heavily on haptic (force and tactile) feedback to manipulate and explore objects. However, in virtual and augmented reality environments, haptic sensations must be artificially generated. Advances in both sensing and feedback are needed to create more realistic virtual scenarios for human interaction, education, and training. This research will implement wearable fingertip haptic feedback systems that enhance human perception and task performance in virtual and augmented reality, and will provide the field of computer-mediated haptics with important new design and rendering insights for a promising form of wearable haptic device. Successful development of fingertip haptic devices and improved tracking methods will lower the cost of integrating haptic technology into virtual reality, and yield more realistic and compelling virtual experiences. Effective haptic feedback systems for virtual reality utilizing the developed technologies will have broad impact by improving human health and well-being through a myriad of applications such as training for critical tasks like surgery and defusing of explosive ordnance, tactile communication to enable design and e-commerce, and immersion in virtual worlds for enhanced education and training. Project outcomes will be disseminated through demonstrations of educational applications, expansion of an online course on haptics, and publicly available software and data. Outreach programs, public lab tours, and mentoring of female and minority graduate students, undergraduates, and high school students will broaden participation of underrepresented groups in engineering.This project has three main technical components. First, instrumented objects will be developed that measure force interactions and integrate that data with external measurements of finger pose. Second, new wearable haptic devices will be developed that use a combination of local kinesthetic feedback and skin deformation to provide realistic haptic feedback with minimal encumbrance. Modular haptic device designs will facilitate rapid testing of various design choices. Then the basic design will be optimized in terms of control strategy, degrees of freedom, skin contact conditions, and finger grounding to best match the real-world grasping and manipulation data collected earlier. Third, predictive tracking algorithms will be developed for grasping based on characteristic behaviors observed in real-world grasping and manipulation data. All of the haptic devices and tracking methods will be evaluated both in virtual and in augmented reality environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在真实的世界中，人们在很大程度上依赖于触觉（力和触觉）反馈来操纵和探索物体。 然而，在虚拟和增强现实环境中，触觉感觉必须人工生成。 需要在传感和反馈方面取得进展，以便为人类互动、教育和培训创建更逼真的虚拟场景。 这项研究将实现可穿戴指尖触觉反馈系统，增强人类的感知和虚拟现实和增强现实中的任务性能，并将提供重要的新设计和渲染的见解，可穿戴触觉设备的一个有前途的形式的计算机为媒介的触觉领域。 指尖触觉设备和改进的跟踪方法的成功开发将降低将触觉技术集成到虚拟现实中的成本，并产生更逼真和更引人注目的虚拟体验。 有效的触觉反馈系统的虚拟现实利用开发的技术将有广泛的影响，通过改善人类的健康和福祉，通过无数的应用程序，如培训的关键任务，如手术和拆除爆炸性弹药，触觉通信，使设计和电子商务，并沉浸在虚拟世界中加强教育和培训。 项目成果将通过教育应用示范、扩大触觉学在线课程以及公开提供的软件和数据传播。 推广计划、公共实验室图尔斯参观以及对女性和少数民族研究生、本科生和高中生的指导将扩大代表性不足的群体对工程的参与。 首先，将开发测量力的相互作用并将该数据与手指姿势的外部测量相结合的仪器化对象。 第二，将开发新的可穿戴触觉设备，其使用局部动觉反馈和皮肤变形的组合来提供具有最小反射的真实触觉反馈。 模块化触觉装置设计将促进各种设计选择的快速测试。 然后，基本设计将在控制策略、自由度、皮肤接触条件和手指接地方面进行优化，以最好地匹配先前收集的真实世界的抓取和操纵数据。 第三，预测跟踪算法将根据在现实世界的抓取和操纵数据中观察到的特征行为来开发抓取。 所有的触觉设备和跟踪方法都将在虚拟和增强现实环境中进行评估。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Estimating Motion Codes from Demonstration Videos

• DOI: 10.1109/iros45743.2020.9341065
• 发表时间: 2020-07
• 期刊: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Maxat Alibayev;D. Paulius;Yu Sun

Multi-Object Grasping - Types and Taxonomy

多对象抓取 - 类型和分类

• DOI: 10.1109/icra46639.2022.9812388
• 发表时间: 2022
• 期刊: 2022 International Conference on Robotics and Automation (ICRA
• 作者: Sun, Yu;Amatova, Eliza;Chen, Tianze
• 通讯作者: Chen, Tianze

A Motion Taxonomy for Manipulation Embedding

• DOI: 10.15607/rss.2020.xvi.045
• 发表时间: 2020-07
• 期刊: ArXiv
• 作者: D. Paulius;Nicholas Eales;Yu Sun

Generalizing Learned Manipulation Skills in Practice

在实践中推广所学的操作技能

• DOI: 10.1109/iros45743.2020.9340739
• 发表时间: 2020
• 期刊: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Wilches, Juan;Huang, Yongqiang;Sun, Yu
• 通讯作者: Sun, Yu

Robot Gaining Accurate Pouring Skills through Self-Supervised Learning and Generalization

• DOI: 10.1016/j.robot.2020.103692
• 发表时间: 2020-11
• 期刊: ArXiv
• 作者: Yongqiang Huang;Juan Wilches;Yu Sun