RI: Medium: Collaborative Research: Closed Loop Perceptual Planning for Dynamic Locomotion

RI：中：协作研究：动态运动的闭环感知规划

• 批准号: 1703319
• 负责人: Kostas Daniilidis
• 金额: $ 32万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1703319&HistoricalAwards=false
• 关键词: RI; Medium; Collaborative; Research; Closed

Modern robots can be seen moving about a variety of terrains and environments, using wheels, legs, and other means, engaging in life-like hopping, jumping, walking, crawling, and running. They execute motions called gaits. An example of a gait is a horse trotting or galloping. Likewise, humans execute walking, running and skipping gaits. Essentially, for either a biological or mechanical systems, a gait is a locomotion pattern that involves large-amplitude body oscillations. Naturally, these motions cause impacts with terrain that jostle on-board perceptual systems and directly influence what the robots actually "see" as they move. For instance, the body motion of a bounding horse-like robot may result in significant occlusions and oscillations in on-board camera systems that confound motion estimation and perceptual feedback.Focusing on complex mobility robots, this project seeks to better understand the coupling between locomotion and visual perception to improve perceptual feedback for closed-loop motion estimation. The work is organized around two key questions: 1) How should a robot look to move well? 2) How should a robot move to see well? To address the first challenge, the periodic structure of gait-based motions will be leveraged to improve perceptual filtering as the robot carries out fixed (pre-determined) motions. The second half of the project will derive perceptual objectives and a new perceptual gait design framework to guide how high degree-of-freedom, complex mobility robots should move (locomote). The goal is to optimize feedback for closed-loop motion implementation, on-line adaptation, and learning, which are currently difficult or impossible for many complex mobility robots.

可以看到现代机器人在各种地形和环境中移动，使用轮子，腿和其他手段，从事像生活一样的跳跃，跳跃，行走，爬行和跑步。它们执行的运动称为步态。步态的一个例子是马小跑或疾驰。同样，人类执行步行，跑步和跳跃步态。基本上，对于生物或机械系统，步态是涉及大幅度身体振荡的运动模式。自然地，这些运动会对地形产生影响，从而挤压机载感知系统，并直接影响机器人在移动时实际“看到”的东西。 例如，一个跳跃的马一样的机器人的身体运动可能会导致显着的遮挡和振荡的车载摄像机系统，混淆运动估计和感知反馈。本项目着眼于复杂的移动机器人，旨在更好地了解运动和视觉感知之间的耦合，以改善闭环运动估计的感知反馈。这项工作是围绕两个关键问题组织的：1）机器人应该如何看起来移动良好？2)机器人应该如何移动才能看得清楚？为了解决第一个挑战，将利用基于步态的运动的周期性结构来改善机器人执行固定（预定）运动时的感知过滤。 该项目的后半部分将推导出感知目标和一个新的感知步态设计框架，以指导高自由度，复杂的移动机器人应该如何移动。 我们的目标是优化反馈的闭环运动的实施，在线适应和学习，这是目前困难或不可能的许多复杂的移动机器人。

项目成果

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

• DOI: 10.15607/rss.2018.xiv.062
• 发表时间: 2018-02
• 期刊: ArXiv
• 作者: A. Z. Zhu;Liangzhe Yuan;Kenneth Chaney;Kostas Daniilidis

Event-Based Vision: A Survey

• DOI: 10.1109/tpami.2020.3008413
• 发表时间: 2022-01-01
• 期刊: IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
• 影响因子: 23.6
• 作者: Gallego, Guillermo;Delbruck, Tobi;Scaramuzza, Davide
• 通讯作者: Scaramuzza, Davide

Self-Supervised Optical Flow with Spiking Neural Networks and Event Based Cameras

具有尖峰神经网络和基于事件的相机的自监督光流

• DOI: 10.1109/iros51168.2021.9635975
• 发表时间: 2021
• 期刊: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Chaney, Kenneth;Panagopoulou, Artemis;Lee, Chankyu;Roy, Kaushik;Daniilidis, Kostas
• 通讯作者: Daniilidis, Kostas

Learning Event-based Height from Plane and Parallax

从平面和视差学习基于事件的高度

• 发表时间: 2019
• 期刊: IEEE/RSJ International Conference on Intelligent Robots and Systems
• 作者: Chaney, Kenneth;Zhu, Alex Zihao;Daniilidis, Kostas
• 通讯作者: Daniilidis, Kostas

EventGAN: Leveraging Large Scale Image Datasets for Event Cameras

• DOI: 10.1109/iccp51581.2021.9466265
• 发表时间: 2019-12
• 期刊: 2021 IEEE International Conference on Computational Photography (ICCP)
• 作者: A. Z. Zhu;ZiYun Wang;Kaung Khant;Kostas Daniilidis