Data Driven Predictive Auditory Cues for Safety and Fluency in Human-Robot Interaction

大多数工业和社交机器人对周围环境没有足够的感知，这导致了广泛的伤害。这可能会阻碍流畅和高效的人类-机器人工作。该项目专注于开发、集成和测试一套机器人运动的声音提示，目的是增强人类安全并允许流畅的交互。声音提示是使用算法创建的，这些算法提供了关于机器人当前和未来行动的丰富信息，提醒人类注意潜在的危险，并允许他们准备和调整他们的工作空间。这样的声音提示有望使用不分散注意力的听觉通道来帮助人类计划他们的行动和对机器人行动的反应。该系统基于音乐驱动的机器人动作，使用一种新的音频生成方法，该方法将提供有关机器人运动的信息。所使用的算法是在新创建的具有风险信息的音频片段的数据集上训练的。该系统可以在工业和个人机器人、私人和公共空间中增加人与机器人交互的安全性、流畅性和信任建立，解决制造、培训、教育和其他方面的任务。为了解决这一目标，该项目分为四个阶段：阶段1-新创建的音频片段数据集的收集、分析、标记和特征提取。第二阶段-开发一种新的神经网络模型，该模型将在收集的数据集上进行训练，以与标记的机器人运动集相关联。然后，模型的输出将被馈送到神经网络，该神经网络将生成由音乐和手势特征调节的长背景原始音频。阶段3-将生成的音频提示集成到一大组机器人手势中，目标是表示机器人的运动和未来的动作。阶段4--在各种人机交互场景中对发声机器人手势的安全性和流畅性进行的全面评估研究。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Most industrial and social robots are not sufficiently aware of their surroundings, which leads to a wide range of injuries. This can hamper fluent and efficient human-robot work. This project focuses on developing, integrating, and testing a set of sound cues for robotic movements, with the goals of enhancing human safety and allowing fluent interaction. The sound cues are created using algorithms which provide rich information about the robots’ current and future actions, alerting humans to potential hazards, and allowing them to prepare and adjust their work space. Such sound cues bear the promise of using aa non-distracting auditory channel to help humans plan their actions and responses to robotic actions. The system is based on music-driven robotic maneuvers using a novel audio generation method that will provide information about the robotic movements. The algorithm used is trained on a newly created dataset of audio clips with risk information. The system can increase safety, fluency and trust building in human-robot interaction in industrial and personal robots, private and public spaces, addressing tasks in manufacturing, training, education, and others.To address this goal the project is divided into four phases: Phase 1 - collection, analysis, labeling and feature extraction of a newly created dataset of audio clips. Phase 2 - development of a novel neural network model that will be trained on the collected dataset in correlation to labeled set of robotic movements. The output of the model will then be fed to a neural network that will generate long-context raw audio conditioned by musical and gestural features. Phase 3 - integration of the generated audio cues into a large set of robotic gestures with the goal of representing robotic motion and future actions. Phase 4 - a comprehensive evaluation study of the sonified robotic gestures for safety and fluency in a variety of human-robot interaction scenarios.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.