NRI: FND: Towards Scalable and Self-Aware Robotic Perception

NRI：FND：迈向可扩展和自我意识的机器人感知

• 批准号: 1924937
• 负责人: Nuno Vasconcelos
• 金额: $ 75万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1924937&HistoricalAwards=false
• 关键词: NRI; FND; Towards; Scalable; Self

Robot vision systems should be fast, to enhance the reaction times of robots to events in the visual world, capable of solving multiple vision problems simultaneously, and aware of their limitations. These properties are critical for robotic safety and collaboration. Safety is enhanced by faster reaction times (e.g. a car faster to detect obstacles has more room to stop before hitting them) and self-awareness (e.g., a robot should choose to stop to operate in situations that it deems too hard to be successful in). Collaboration is enhanced by scalability (which allows co-robots to solve more problems and thus behave more like human collaborators) and self-awareness (which simplifies the division of tasks between humans and robots, or teams of robots, with different skills). However, these properties have not been the focus of computer vision research, which has mostly addressed the design of networks that solve single tasks, usually requiring heavy computation and relatively low frame rates, and simply attempt to process all examples without any consideration for how difficult they are. This project addresses all these challenges, laying the foundation for a new generation of robotic perception systems that are more efficient, scalable, and self-aware. The research has applicability in areas of societal relevance, such as manufacturing, self-driving vehicles, intelligent systems, assisted living, homeland security, etc. Educationally, the project will provide exciting opportunities for both graduate and undergraduate research.This project pursues a research agenda composed of several integrated contributions that advance the state of the art in deep learning for robotic vison. This includes 1) novel neural network quantization techniques that address the quantization of both network weights and activations, leading to deep learning models that can be fully implemented with binary operations, significantly enhancing the speed of all AI computations; 2) new families of networks that exploit extensive parameter sharing to achieve scalable inference for task ecologies, substantially increasing the number of networks that can be cached in a processor and, therefore, the number of vision problems that can be solved simultaneously by a robot; 3) new network architectures for self-aware deep learning, capable of assessing the difficulty of each example, predicting failures, and refusing to process examples that are too difficult, so as to mitigate the possibility of catastrophic errors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人视觉系统应该快速，以提高机器人对视觉世界中事件的反应时间，能够同时解决多个视觉问题，并意识到它们的局限性。这些特性对于机器人的安全和协作至关重要。更快的反应时间（例如，更快地检测到障碍物的汽车在撞击它们之前有更多的空间停下来）和自我意识（例如，机器人应该选择在它认为太难成功的情况下停止操作）。协作通过可扩展性（允许协作机器人解决更多问题，从而表现得更像人类协作者）和自我意识（简化人类和机器人或机器人团队之间的任务分工，具有不同的技能）得到增强。然而，这些特性并不是计算机视觉研究的重点，计算机视觉研究主要解决单个任务的网络设计，通常需要大量的计算和相对较低的帧速率，并且简单地尝试处理所有示例，而不考虑它们的难度。该项目解决了所有这些挑战，为新一代机器人感知系统奠定了基础，这些系统更有效，可扩展和自我意识。该研究在社会相关领域具有适用性，如制造业，自动驾驶汽车，智能系统，辅助生活，国土安全等。在教育方面，该项目将为研究生和本科生提供令人兴奋的研究机会。该项目追求的研究议程由几个综合贡献组成，这些贡献推动了机器人视觉深度学习的最新发展。这包括1）新颖的神经网络量化技术，解决了网络权重和激活的量化问题，从而产生了可以完全用二进制运算实现的深度学习模型，显著提高了所有AI计算的速度; 2）利用广泛的参数共享来实现任务生态的可扩展推理的新的网络族，实质上增加了可以缓存在处理器中的网络的数量，因此增加了机器人可以同时解决的视觉问题的数量; 3）用于自我感知深度学习的新网络架构，能够评估每个示例的难度，预测失败，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier

使用深度现实分类器解决长尾识别问题

• 发表时间: 2020
• 期刊: European Conference on Computer Vision
• 作者: Tz-Ying Wu, Pedro Morgado

Learning of Visual Relations: The Devil is in the Tails

• DOI: 10.1109/iccv48922.2021.01512
• 发表时间: 2021-08
• 期刊: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: Alakh Desai;Tz-Ying Wu;Subarna Tripathi;N. Vasconcelos

Audio-Visual Instance Discrimination with Cross-Modal Agreement

• DOI: 10.1109/cvpr46437.2021.01229
• 发表时间: 2020-04
• 期刊: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Pedro Morgado;N. Vasconcelos;Ishan Misra

Breadcrumbs: Adversarial Class-Balanced Sampling for Long-tailed Recognition

• DOI: 10.1007/978-3-031-20053-3_37
• 发表时间: 2021-05
• 期刊: ArXiv
• 作者: Bo Liu;Haoxiang Li;Hao Kang;G. Hua;N. Vasconcelos

Background Data Resampling for Outlier-Aware Classification

用于异常值感知分类的背景数据重采样

• 发表时间: 2020
• 期刊: IEEE Conference on Computer Vision and Pattern Recognition
• 作者: Yi Li, Nuno Vasconcelos