RI: Medium: Collaborative Research: Closed Loop Perceptual Planning for Dynamic Locomotion

RI：中：协作研究：动态运动的闭环感知规划

• 批准号: 1703319
• 负责人: Kostas Daniilidis
• 金额: $ 32万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1703319&HistoricalAwards=false
• 关键词: RI; Medium; Collaborative; Research; Closed

Modern robots can be seen moving about a variety of terrains and environments, using wheels, legs, and other means, engaging in life-like hopping, jumping, walking, crawling, and running. They execute motions called gaits. An example of a gait is a horse trotting or galloping. Likewise, humans execute walking, running and skipping gaits. Essentially, for either a biological or mechanical systems, a gait is a locomotion pattern that involves large-amplitude body oscillations. Naturally, these motions cause impacts with terrain that jostle on-board perceptual systems and directly influence what the robots actually "see" as they move. For instance, the body motion of a bounding horse-like robot may result in significant occlusions and oscillations in on-board camera systems that confound motion estimation and perceptual feedback.Focusing on complex mobility robots, this project seeks to better understand the coupling between locomotion and visual perception to improve perceptual feedback for closed-loop motion estimation. The work is organized around two key questions: 1) How should a robot look to move well? 2) How should a robot move to see well? To address the first challenge, the periodic structure of gait-based motions will be leveraged to improve perceptual filtering as the robot carries out fixed (pre-determined) motions. The second half of the project will derive perceptual objectives and a new perceptual gait design framework to guide how high degree-of-freedom, complex mobility robots should move (locomote). The goal is to optimize feedback for closed-loop motion implementation, on-line adaptation, and learning, which are currently difficult or impossible for many complex mobility robots.

可以看到现代机器人在各种地形和环境中移动，使用轮子、腿和其他方式，从事类似生活的跳跃、跳跃、行走、爬行和跑步。它们执行被称为步态的动作。步态的一个例子是马在小跑或奔跑。同样，人类也执行行走、奔跑和跳跃的步态。从本质上讲，无论是生物系统还是机械系统，步态都是一种涉及到大幅度身体振荡的运动模式。自然，这些运动会与地形产生碰撞，这些地形会碰撞机载感知系统，并直接影响机器人在移动时实际“看到”的内容。例如，弹跳类马机器人的身体运动可能会导致机载摄像系统中严重的遮挡和振荡，混淆了运动估计和感知反馈。本课题着眼于复杂的移动机器人，试图更好地了解运动和视觉感知之间的耦合，以改善闭环系统运动估计的感知反馈。这项工作围绕两个关键问题展开：1)机器人看起来应该如何移动才能看起来很好？2)机器人应该如何移动才能看起来很好？为了解决第一个挑战，将利用基于步态的运动的周期性结构来改进感知过滤，因为机器人执行固定(预先确定的)运动。该项目的后半部分将得出感知目标和新的感知步态设计框架，以指导高自由度、复杂的移动机器人应该如何移动(运动)。目标是优化反馈以实现闭环运动执行、在线自适应和学习，这些目前对于许多复杂的移动机器人来说是困难或不可能的。

项目成果

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

• DOI: 10.15607/rss.2018.xiv.062
• 发表时间: 2018-02
• 期刊: ArXiv
• 作者: A. Z. Zhu;Liangzhe Yuan;Kenneth Chaney;Kostas Daniilidis

Event-Based Vision: A Survey

• DOI: 10.1109/tpami.2020.3008413
• 发表时间: 2022-01-01
• 期刊: IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
• 影响因子: 23.6
• 作者: Gallego, Guillermo;Delbruck, Tobi;Scaramuzza, Davide
• 通讯作者: Scaramuzza, Davide

Self-Supervised Optical Flow with Spiking Neural Networks and Event Based Cameras

具有尖峰神经网络和基于事件的相机的自监督光流

• DOI: 10.1109/iros51168.2021.9635975
• 发表时间: 2021
• 期刊: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Chaney, Kenneth;Panagopoulou, Artemis;Lee, Chankyu;Roy, Kaushik;Daniilidis, Kostas
• 通讯作者: Daniilidis, Kostas

Learning Event-based Height from Plane and Parallax

从平面和视差学习基于事件的高度

• 发表时间: 2019
• 期刊: IEEE/RSJ International Conference on Intelligent Robots and Systems
• 作者: Chaney, Kenneth;Zhu, Alex Zihao;Daniilidis, Kostas
• 通讯作者: Daniilidis, Kostas

EventGAN: Leveraging Large Scale Image Datasets for Event Cameras

• DOI: 10.1109/iccp51581.2021.9466265
• 发表时间: 2019-12
• 期刊: 2021 IEEE International Conference on Computational Photography (ICCP)
• 作者: A. Z. Zhu;ZiYun Wang;Kaung Khant;Kostas Daniilidis