NRI: FND: Action-perception loops over 5G millimeter wave wireless for cooperative manipulation

随着执行操纵或腿部移动任务的机器人的自主性增加，它们需要不断增长的计算资源来成功地感知环境并做出决策。然而，移动机器人从根本上受到重量、形状和动力自主性的限制，这对它们可以携带的计算能力施加了重要限制。作为这种困境的可能解决方案，云机器人旨在将计算转移到远程服务器，但到目前为止，对于机器人和云之间需要低延迟通信和高数据带宽的任务，它仍然是一种难以捉摸的方法。这类任务包括控制、规划和感知算法，用于对象操纵和腿部运动。第五代蜂窝网络技术(5G)可能会给云机器人带来革命性的变化，因为它有望获得前所未有的高带宽和低延迟无线通信。然而，在确保通信可靠性、通信退化情况下机器人操作的安全性以及多机器人系统的可扩展性方面，仍然存在着巨大的挑战。该项目旨在将5G技术完全融入执行复杂操作和移动任务的机器人系统。它将开发新的感知、控制和规划算法，在机器人和云之间最佳地分配计算，以保证机器人的安全操作。最终，这些算法将加快5G机器人在人类环境中的无处不在的部署，并开启医疗保健、服务和工业机器人的广泛应用。该项目采用整体控制、感知和通信的方法，为自主机器人基于边缘的无线实时动作感知回路奠定基础。它是按照四个主要的研究方向组织的。首先，它将研究在通信质量下降的情况下，分布在网络边缘和机器人之间的具有性能保证的新型最优控制和规划算法。其次，它将提出高效的计算划分技术，用于使用具有高数据速率的多模式感知的实时感知。第三，它将通过实验和模拟来表征机器人环境中的5G特定通信通道，并设计新的毫米波通信协议，这些协议是为实时机器人动作感知环路量身定做的。最后，对单机器人和多机器人系统的广泛实验，包括固定和移动机械手以及四足机器人，将展示支持5G的机器人系统的独特能力。该项目的外联活动将有助于降低科学家和行业的准入门槛，这些科学家和行业寻求通过开源分发算法和通过纽约大学无线传播结果来利用5G支持的机器人。这一努力将有助于本科生和研究生的教育，利用其成果进行课程开发，并提供有监督的项目，有可能直接在最先进的实验平台上工作。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

As the autonomy of robots performing manipulation or legged locomotion tasks increases, they require an ever-growing amount of computational resources to successfully perceive their environment and make decisions. Yet, mobile robots are fundamentally constrained by weight, shape and power autonomy which impose important limits on the computational capabilities they can carry. As a possible answer to such dilemma, cloud robotics aims to move computation to remote servers, but it has thus far remained an elusive approach for tasks that necessitate low delay communication and high data bandwidth between the robot and the cloud. Such tasks include control, planning and perception algorithms for object manipulation and legged locomotion. The 5th generation of cellular network technology (5G) could revolutionize cloud robotics as it promises unprecedented access to high bandwidth and low latency wireless communication. Yet, formidable challenges remain to ensure communication reliability, safety of robotic operation under communication degradation, and scalability to multi-robot systems. This project aims to fully incorporate 5G technology into robotics systems performing complex manipulation and locomotion tasks. It will develop novel perception, control and planning algorithms that optimally distribute computations between robots and the cloud for guaranteed safe robotic operation. Ultimately, these algorithms will accelerate the ubiquitous deployment of untethered 5G-enabled robots in human environments and unlock a large range of applications for healthcare, service and industrial robotics.The project takes a holistic approach to control, perception and communication to establish the foundations of edge-based wireless real-time action-perception loops for autonomous robots. It is organized along four main thrusts of research. First, it will investigate novel optimal control and planning algorithms distributed between the network edge and the robot with performance guarantees under communication degradation. Second, it will propose efficient computational partitioning techniques for real-time perception using multi-modal sensing with high data rates. Third, it will characterize 5G specific communication channels in a robotics environment via experiments and simulations and design new mmWave communication protocols tailored for real-time robotic action-perception loops. Finally, extensive experiments on single and multi-robot systems, including fixed and mobile manipulators and a quadruped robot, will demonstrate the unique capabilities of 5G-enabled robotic systems. The outreach activities of the project will contribute to lowering barriers to entry for scientists and industries that seek to exploit 5G-enabled robotics through open-source distribution of algorithms and dissemination of results via NYU WIRELESS. This effort will contribute to the education of undergraduate and graduate students, leveraging its outcomes for curriculum development and offering supervised projects with the possibility to work directly on state-of-the-art experimental platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.