权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

EAGER: Deep Architectures for Ppredicting 3D Object Motion

EAGER：预测 3D 对象运动的深层架构

基本信息

批准号：
1942069
负责人：
Evangelos Kalogerakis
金额：
$ 17.54万
依托单位：
University of Massachusetts Amherst
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-09-01 至 2021-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942069&HistoricalAwards=false
关键词：
EAGER Deep Architectures Ppredicting 3D

项目摘要

Our everyday living environments are populated with lots of functional objects with which we can interact through their moving parts (e.g., swivel chairs, laptops, bikes and cars, to name just a few). For autonomous agents to correctly interact with these objects in real-world settings, the agents must be equipped with algorithms that are able to parse the objects into their moving parts. But that is not enough. Through the widespread use of commodity 3D sensors and modern 3D modeling techniques, large repositories (such as ShapeNet) containing millions of digital representations of everyday objects are now available, but these representations are for the most part currently static, that is to say they represent single snapshots of objects. To make use of these object representations in dynamic, virtual environments and in animation applications, methods that automatically segment them into moving parts and synthesize plausible motions for them are needed. This project will explore the design, implementation, and testing of new deep learning architectures that accomplish this, and thereby bring large portions of static 3D datasets "to life." The new algorithms will have broad industrial impact by advancing 3D modeling and animation software, while the generated motion data will be useful for training new computer vision algorithms for object motion recognition and tracking in videos.Achieving the project goals will require development of new algorithms to convert static digital representations of 3D objects into dynamic ones by automatically recognizing their moving parts and animating them based on input reference videos of similar objects from the real world and through incorporation of novel methods for estimating partial 2D-3D correspondences, for lifting 2D motion cues to 3D, and for inferring motion rigs for 3D shapes. The project will be organized into two main thrusts, each of which will present its own research challenges. The first thrust will investigate new deep learning architectures for performing mobility-based segmentation of 3D objects and predicting the underlying motion of their parts. The architecture will be applied to man-made objects with rigidly moving parts. This part of the research will be carried out in the first year of the project. The second thrust will extend the previous work to animate 3D models representing living organisms such as quadrupeds, birds and fish (i.e., animals from the DigitalLife dataset). These models undergo non-rigid deformations, so the architecture will have to be modified to estimate and control more sophisticated deformation primitives. This part of the work will be executed in the second year of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

我们的日常生活环境中充满了许多功能性物体，我们可以通过它们的移动部件（例如转椅、笔记本电脑、自行车和汽车等）进行交互。为了让自主代理在现实世界中正确地与这些对象进行交互，代理必须配备能够将对象解析为移动部分的算法。但这还不够。通过商品 3D 传感器和现代 3D 建模技术的广泛使用，现在可以使用包含数百万个日常对象的数字表示的大型存储库（例如 ShapeNet），但这些表示目前大部分是静态的，也就是说它们表示对象的单个快照。为了在动态、虚拟环境和动画应用中使用这些对象表示，需要自动将它们分割成运动部件并为其合成合理运动的方法。该项目将探索实现这一目标的新深度学习架构的设计、实现和测试，从而使大部分静态 3D 数据集“栩栩如生”。新算法将通过推进 3D 建模和动画软件产生广泛的工业影响，而生成的运动数据将有助于训练新的计算机视觉算法，以进行视频中的物体运动识别和跟踪。实现项目目标需要开发新算法，通过自动识别 3D 对象的运动部分并根据来自现实世界的类似对象的输入参考视频和通过结合了估计部分 2D-3D 对应关系、将 2D 运动线索提升到 3D 以及推断 3D 形状的运动装备的新颖方法。该项目将分为两个主要方向，每个方向都将提出自己的研究挑战。第一个重点将研究新的深度学习架构，用于执行基于移动性的 3D 对象分割并预测其部件的基本运动。该架构将应用于具有刚性移动部件的人造物体。这部分研究将在项目的第一年进行。第二个重点是将之前的工作扩展到代表生物体的 3D 模型动画，例如四足动物、鸟类和鱼类（即来自 DigitalLife 数据集的动物）。这些模型经历非刚性变形，因此必须修改架构以估计和控制更复杂的变形基元。这部分工作将在项目的第二年执行。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，认为值得支持。