Data Driven Predictive Auditory Cues for Safety and Fluency in Human-Robot Interaction

大多数工业和社交机器人对周围环境没有足够的意识，这导致了大范围的伤害。这可能会妨碍流畅和高效的人机工作。这个项目的重点是开发、集成和测试一套机器人运动的声音提示，目的是提高人类的安全，并允许流畅的互动。声音线索是通过算法创建的，该算法提供了关于机器人当前和未来行动的丰富信息，提醒人类注意潜在的危险，并允许他们准备和调整他们的工作空间。这种声音线索有望利用一种不分散注意力的听觉渠道，帮助人类计划自己的行动，并对机器人的行动做出反应。该系统基于音乐驱动的机器人动作，使用一种新颖的音频生成方法，该方法将提供有关机器人运动的信息。使用的算法是在一个新创建的带有风险信息的音频片段数据集上进行训练的。该系统可以提高工业和个人机器人、私人和公共空间人机交互的安全性、流畅性和信任度，解决制造、培训、教育等领域的任务。为了实现这一目标，该项目分为四个阶段：第一阶段——收集、分析、标记和特征提取新创建的音频片段数据集。第二阶段-开发一种新的神经网络模型，该模型将在收集的数据集上进行训练，并与标记的机器人运动集相关。然后，模型的输出将被馈送到一个神经网络，该网络将生成由音乐和手势特征调节的长上下文原始音频。阶段3 -将生成的音频线索集成到大量机器人手势中，目标是表示机器人的运动和未来的动作。阶段4 -在各种人机交互场景中，对机器人手势的安全性和流畅性进行综合评估研究。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Most industrial and social robots are not sufficiently aware of their surroundings, which leads to a wide range of injuries. This can hamper fluent and efficient human-robot work. This project focuses on developing, integrating, and testing a set of sound cues for robotic movements, with the goals of enhancing human safety and allowing fluent interaction. The sound cues are created using algorithms which provide rich information about the robots’ current and future actions, alerting humans to potential hazards, and allowing them to prepare and adjust their work space. Such sound cues bear the promise of using aa non-distracting auditory channel to help humans plan their actions and responses to robotic actions. The system is based on music-driven robotic maneuvers using a novel audio generation method that will provide information about the robotic movements. The algorithm used is trained on a newly created dataset of audio clips with risk information. The system can increase safety, fluency and trust building in human-robot interaction in industrial and personal robots, private and public spaces, addressing tasks in manufacturing, training, education, and others.To address this goal the project is divided into four phases: Phase 1 - collection, analysis, labeling and feature extraction of a newly created dataset of audio clips. Phase 2 - development of a novel neural network model that will be trained on the collected dataset in correlation to labeled set of robotic movements. The output of the model will then be fed to a neural network that will generate long-context raw audio conditioned by musical and gestural features. Phase 3 - integration of the generated audio cues into a large set of robotic gestures with the goal of representing robotic motion and future actions. Phase 4 - a comprehensive evaluation study of the sonified robotic gestures for safety and fluency in a variety of human-robot interaction scenarios.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.