Unified Vision-Based Motion Estimation and Control for Multiple and Complex Robots

该项目使机器人团队能够在物理任务上进行协作，例如在人类工人的指导下用预制构件组装建筑物。在这样的环境中，每个机器人可能都配备了摄像头来定位自己，并且对它的移动方式有一些限制。 为了实现机器人团队的目标，每个机器人都需要知道自己的位置，哪里可以找到预制构件，以及最终的建筑应该放在哪里。此外，机器人还需要相互协调，以确定它们如何移动来运输和组装组件，并检查它们的工作结果。该项目开发了新的数学和工程方法，使机器人团队能够协同工作，以实现他们的目标。所开发的方法将是可推广的，使它们适用于许多不同的情况，并获得比现有方法更好的结果。为了促进这项工作产生更广泛的影响，研究小组将开发一个易于使用的软件，以促进将研究应用于不同的和新的问题。研究团队还将与Autodesk Inc.的工程师合作。以确保所开发的解决方案可以使当前工业中使用的现有专业设计和可视化工具受益。上面列出的机器人任务意味着非平凡的视觉-Kinodynamic约束（例如，旋转、透视投影），其源自机器人如何移动和感知的固有几何特性。该项目引入了一种新的参数化问题，称为运动形状，可以灵活地将视觉Kinodynamic约束（如闭合运动链，多个图像之间的特征匹配，投影和视场约束）的不同组合作为低秩矩阵上的线性约束。相关的优化求解器，基于交替方向乘法，将通过在低秩投影和最小二乘步骤之间进行迭代，以优雅和统一的方式找到解决方案;然后将不同的应用特定要求简单地作为线性约束纳入求解器中。所开发的技术的主要智力优点在于其多功能性和全面解决传统上使用单独算法堆栈解决的复杂问题的能力（例如，匹配图像中的特征，然后是场景的三维重建和机器人的定位，然后是在运动学约束下在重建的地图中进行规划）。该项目得到机器人技术跨部门基础研究计划的支持，由工程局（ENG）和计算机与信息科学与工程局（CISE）共同管理和资助该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The project enables teams of robots to collaborate on physical tasks, such as assembling a building from prefabricated components under the direction of a human worker. In such settings, each robot might be equipped with cameras to orient itself and have some limitations on how it can move. To achieve the robotic team’s goals, each robot needs to know its location, where to find the prefabricated components, and where the final building should be placed. In addition, the robots need to coordinate with each other on how they move to transport and assemble the components, and to inspect the results of their work. This project develops novel mathematical and engineering methods that would enable the robotic teams to work collaboratively to achieve their goals. The developed methods will be generalizable making them applicable to many different situations and obtaining better results than what is possible with existing approaches. To facilitate a broader impact of this work, the research team will develop an easy-to-use software that facilitates the application of the research to different and new problems. The research team will also collaborate with engineers from Autodesk Inc. to ensure that the developed solution can benefit existing professional design and visualization tools currently used in industry.The robot tasks lay out above imply non-trivial vision-kinodynamic constraints (e.g., rotations, perspective projections) deriving from the intrinsic geometric properties of how the robots move and sense. This project introduces a novel parametrization of the problem, called Shape-of-Motion, that can flexibly incorporate different combinations of vision-kinodynamic constraints (such as close-kinematic-chain, feature matching across multiple images, projection, and field-of-view constraints) as linear constraints on a low-rank matrix. An associated optimization solver, based on the Alternating Direction Method of Multipliers, will find solutions in an elegant and unified manner by iterating between low-rank projections and least-squares steps; different application-specific requirements will then be simply incorporated in the solver as linear constraints. The main intellectual merit of the developed technique is in its versatility and ability to holistically tackle complex problems that are traditionally solved using stacks of separate algorithms (e.g., matching features across images, followed by a 3-D reconstruction of the scene and localization of the robots, followed by planning in the reconstructed map under kinematic constraints).This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.