Planning and executing robotic actions using simulated image sequences created by generative deep neural networks

使用生成深度神经网络创建的模拟图像序列来规划和执行机器人动作

基本信息

项目摘要

Common methods for robotic action planning rely on symbolic processing. For example: AI planning requires domain knowledge and planning algorithms. Similarly, physics simulations, on which planning can be based too, have to rely on explicitly encoded programming instructions. Different from this, the goal of this work is to create a computer vision-based system that is able to generate short plans for robotic manipulation actions based on implicitly simulated image sequences. To this end, we will first determine different selected manipulation affordances in a visual scene using an encoder-decoder DNN (deep neural network). The resulting ‘affordance map’, thus, represents permissive action preconditions. Then we will use a different, generative DNN, which takes the scene as input and creates a new 3D-output scene by “imagining” one of the manipulation actions for which an affordance exists. Hence, this new scene shows how the situation would change if this action would actually be performed. As this is done in 3D, the resulting scene can be checked for geometric consistency and – if ok – the system has very likely arrived at a permitted action post-condition. Using this ‘imagined’ output scene for a next action, short planning sequences will be generated, which are then executed with our robots. This allows for rigorous quantification of real action outcome against “imagined” outcome. The central hypothesis that underlies this work is that it should be possible to create (short) executable plans relying entirely on sub-symbolic information. The main benefit of this is that simple everyday tasks for service robots might become plannable with a much reduced effort for explicit representations.
机器人动作规划的常用方法依赖于符号处理。例如:AI规划需要领域知识和规划算法。类似地,规划也可以基于物理模拟,必须依赖于明确编码的编程指令。与此不同的是,这项工作的目标是创建一个基于计算机视觉的系统,该系统能够生成基于隐式模拟图像序列的机器人操作动作的短计划。为此,我们将首先使用编码器-解码器DNN(深度神经网络)确定视觉场景中不同的选定操作启示。由此产生的“示能图”,因此,表示许可的行动的先决条件。然后,我们将使用一个不同的生成DNN,它将场景作为输入,并通过“想象”一个存在启示的操作动作来创建一个新的3D输出场景。因此,这个新场景显示了如果实际执行该动作,情况将如何变化。由于这是在3D中完成的,因此可以检查所产生的场景的几何一致性,并且如果可以,则系统很可能已经达到允许的动作后置条件。 使用这个“想象的”输出场景进行下一个动作,将生成简短的规划序列,然后由我们的机器人执行。这允许严格量化真实的行动结果与“想象的”结果。这项工作的核心假设是,它应该是可以创建(短)可执行的计划完全依赖于子符号信息。这样做的主要好处是,服务机器人的简单日常任务可能变得可规划,而显式表示的工作量大大减少。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Professor Dr. Florentin Wörgötter其他文献

Professor Dr. Florentin Wörgötter的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

相似海外基金

Collaborative Research: CISE: Large: Executing Natural Instructions in Realistic Uncertain Worlds
合作研究:CISE:大型:在现实的不确定世界中执行自然指令
  • 批准号:
    2321852
  • 财政年份:
    2023
  • 资助金额:
    --
  • 项目类别:
    Continuing Grant
Collaborative Research: CISE: Large: Executing Natural Instructions in Realistic Uncertain Worlds
合作研究:CISE:大型:在现实的不确定世界中执行自然指令
  • 批准号:
    2321851
  • 财政年份:
    2023
  • 资助金额:
    --
  • 项目类别:
    Continuing Grant
Genetic requirements for executing SUMO stress signals and achieving stress tolerance
执行 SUMO 应激信号和实现应激耐受性的遗传要求
  • 批准号:
    10514836
  • 财政年份:
    2022
  • 资助金额:
    --
  • 项目类别:
Multi-functional robot control platform, executing multi-stage process files in complex, non-deterministic environments, achieving 99.8% performance level reliability in accordance to guaranteed functional outcome predefined by sensoric data
多功能%20机器人%20控制%20平台,%20执行%20多阶段%20进程%20文件%20in%20复杂,%20非确定性%20环境,%20实现%2099.8%%20性能%20级别%20可靠性%20in%20根据%20to%
  • 批准号:
    10024275
  • 财政年份:
    2022
  • 资助金额:
    --
  • 项目类别:
    Collaborative R&D
York - Birmingham Contemporary Music Group - Identifying, systematising, executing, and communicating core skills in contemporary music performance
约克 - 伯明翰当代音乐小组 - 识别、系统化、执行和交流当代音乐表演的核心技能
  • 批准号:
    2446556
  • 财政年份:
    2020
  • 资助金额:
    --
  • 项目类别:
    Studentship
Application Title: Development of a working prototype of a tokenised data-security system that keeps tokenised data at rest secure when executing queries and searches to prevent fraud and give protection against cyber attacks
申请标题:开发代币化数据安全系统的工作原型,该系统在执行查询和搜索时保持静态代币化数据的安全,以防止欺诈并提供针对网络攻击的保护
  • 批准号:
    88375
  • 财政年份:
    2020
  • 资助金额:
    --
  • 项目类别:
    Collaborative R&D
Collaborative Research: Frameworks: Production quality Ecosystem for Programming and Executing eXtreme-scale Applications (EPEXA)
合作研究:框架:用于编程和执行超大规模应用程序的生产质量生态系统 (EPEXA)
  • 批准号:
    1931384
  • 财政年份:
    2019
  • 资助金额:
    --
  • 项目类别:
    Standard Grant
Collaborative Research: Frameworks: Production quality Ecosystem for Programming and Executing eXtreme-scale Applications (EPEXA)
合作研究:框架:用于编程和执行超大规模应用程序的生产质量生态系统 (EPEXA)
  • 批准号:
    1931387
  • 财政年份:
    2019
  • 资助金额:
    --
  • 项目类别:
    Standard Grant
Collaborative Research: Frameworks: Production quality Ecosystem for Programming and Executing eXtreme-scale Applications (EPEXA)
合作研究:框架:用于编程和执行超大规模应用程序的生产质量生态系统 (EPEXA)
  • 批准号:
    1931347
  • 财政年份:
    2019
  • 资助金额:
    --
  • 项目类别:
    Standard Grant
Management taking advantage of ingenuities aged nurses employ to overcome age-related difficulties in executing their duties
管理层利用老年护士的聪明才智克服执行职责时与年龄有关的困难
  • 批准号:
    17K17421
  • 财政年份:
    2018
  • 资助金额:
    --
  • 项目类别:
    Grant-in-Aid for Young Scientists (B)
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了