NRI: Real-Time Semantic Computer Vision for Co-Robotics
NRI:协作机器人的实时语义计算机视觉
基本信息
- 批准号:1637941
- 负责人:
- 金额:$ 71.91万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2016
- 资助国家:美国
- 起止时间:2016-09-01 至 2020-08-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
This project develops real-time object recognition algorithms that generate extensive semantic object descriptions as a side effect of recognition. This side-information includes perceived object costs, attributes (object properties), and affordances (actions afforded by objects). With these, the act of recognizing a "door knob" would automatically produce the information that this is a "flexible" object, "made of metal," which "can be grasped" and "can be twisted," but "cannot be eaten." For robotics, this information is sometimes more important than the recognition of the object itself. The project enables robots to perform zero shot learning, e.g. learn to recognize door knobs by simply being told that these are objects that "are flexible, made of metal, can be grasped and twisted but not eaten." The research has applicability in areas such as manufacturing, intelligent systems, assisted living, and homeland security. Educationally, the project provides an exciting opportunity for undergraduate research.This research develops new methods for top-down (task-driven) regularization of deep learning algorithms, though a combination of structural and loss-based regularizers. Structural regularizers constrain object and scene recognition models to guarantee speed and automatic generation of rich mid-level semantic (MLS) descriptions as a side effect of recognition. Loss-based regularizers penalize errors in the multiple semantic outputs of these models, enabling simultaneously high performance in object recognition, MLS predictions, and zero-shot learning. The resulting learning algorithms will endow robots with human-like abilities to infer rich MLS descriptions of objects and scenes as a "side effect" of object recognition and scene classification, in real-time. These contributions will be developed in the context of a new co-robotics problem, person-following unmanned aerial vehicles, where computer vision plays a mission critical role for tasks such as control and semantic motion planning but whose requirements in terms of speed and MLS inference are far superior to what is feasible today.
该项目开发实时对象识别算法,生成广泛的语义对象描述作为识别的副作用。这些辅助信息包括感知的对象成本、属性(对象属性)和启示(对象提供的动作)。有了这些,识别“门把手”的行为将自动产生这样的信息:这是一个“柔性”物体,“由金属制成”,“可以抓住”和“可以扭曲”,但“不能吃”。“对于机器人来说,这些信息有时比物体本身的识别更重要。该项目使机器人能够进行零射击学习,例如,只需告诉它们这些是“灵活的,由金属制成的,可以抓住和扭曲但不能吃的”物体,就可以学会识别门把手。“这项研究在制造、智能系统、辅助生活和国土安全等领域具有适用性。在教育方面,该项目为本科生研究提供了一个令人兴奋的机会。这项研究开发了自上而下(任务驱动)深度学习算法正则化的新方法,尽管结合了结构正则化器和基于损失的正则化器。结构正则化器约束对象和场景识别模型,以保证速度和自动生成丰富的中级语义(MLS)描述作为识别的副作用。基于损失的正则化器惩罚这些模型的多个语义输出中的错误,同时实现对象识别,MLS预测和零射击学习的高性能。由此产生的学习算法将赋予机器人类似人类的能力,以实时推断对象和场景的丰富MLS描述,作为对象识别和场景分类的“副作用”。这些贡献将在一个新的合作机器人问题的背景下,人以下无人驾驶飞行器,其中计算机视觉起着使命的关键作用,如控制和语义运动规划,但其要求的速度和MLS推理是远远优于上级什么是可行的今天。
项目成果
期刊论文数量(17)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Semantically Consistent Regularization for Zero-Shot Recognition
- DOI:10.1109/cvpr.2017.220
- 发表时间:2017-04
- 期刊:
- 影响因子:0
- 作者:Pedro Morgado;N. Vasconcelos
- 通讯作者:Pedro Morgado;N. Vasconcelos
Catastrophic Child's Play: Easy to Perform, Hard to Defend Adversarial Attacks
- DOI:10.1109/cvpr.2019.00945
- 发表时间:2019-06
- 期刊:
- 影响因子:0
- 作者:Chih-Hui Ho;Brandon Leung;Erik Sandström;Yen Chang;N. Vasconcelos
- 通讯作者:Chih-Hui Ho;Brandon Leung;Erik Sandström;Yen Chang;N. Vasconcelos
Deep Scene Image Classification with the MFAFVNet
- DOI:10.1109/iccv.2017.613
- 发表时间:2017-10
- 期刊:
- 影响因子:0
- 作者:Yunsheng Li;Mandar Dixit;N. Vasconcelos
- 通讯作者:Yunsheng Li;Mandar Dixit;N. Vasconcelos
Object based Scene Representations using Fisher Scores of Local Subspace Projections
- DOI:
- 发表时间:2016-12
- 期刊:
- 影响因子:0
- 作者:Mandar Dixit;N. Vasconcelos
- 通讯作者:Mandar Dixit;N. Vasconcelos
Self-Supervised Generation of Spatial Audio for 360 Video
- DOI:
- 发表时间:2018-09
- 期刊:
- 影响因子:0
- 作者:Pedro Morgado;N. Vasconcelos;Timothy R. Langlois;Oliver Wang
- 通讯作者:Pedro Morgado;N. Vasconcelos;Timothy R. Langlois;Oliver Wang
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Nuno Vasconcelos其他文献
Advanced methods for robust object detection
用于稳健物体检测的先进方法
- DOI:
- 发表时间:
2022 - 期刊:
- 影响因子:0
- 作者:
Zhaowei Cai;Nuno Vasconcelos - 通讯作者:
Nuno Vasconcelos
121 Neural Network Dose Prediction for Cervical Brachytherapy: Overcoming Data Scarcity for Applicator-Specific Models
用于宫颈近距离放射治疗的 121 神经网络剂量预测:克服特定施源器模型的数据稀缺性
- DOI:
10.1016/s0167-8140(23)89212-x - 发表时间:
2023-09-01 - 期刊:
- 影响因子:5.300
- 作者:
Lance Moore;Karoline Kallis;Nuno Vasconcelos;Kelly Kisling;Dominique Rash;Catheryn Yashar;Jyoti Mayadev;Kevin Moore;Sandra Meyers - 通讯作者:
Sandra Meyers
Towards Calibrated Multi-label Deep Neural Networks
迈向校准的多标签深度神经网络
- DOI:
- 发表时间:
- 期刊:
- 影响因子:0
- 作者:
Jiacheng Cheng;Nuno Vasconcelos - 通讯作者:
Nuno Vasconcelos
Nuno Vasconcelos的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Nuno Vasconcelos', 18)}}的其他基金
RI:Small:Dynamic Networks for Efficient, Adaptive, and Multimodal Vision
RI:Small:用于高效、自适应和多模态视觉的动态网络
- 批准号:
2303153 - 财政年份:2023
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
FAI: Towards Holistic Bias Mitigation in Computer Vision Systems
FAI:迈向计算机视觉系统中的整体偏差缓解
- 批准号:
2041009 - 财政年份:2021
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
NRI: FND: Towards Scalable and Self-Aware Robotic Perception
NRI:FND:迈向可扩展和自我意识的机器人感知
- 批准号:
1924937 - 财政年份:2019
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
BIGDATA: Collaborative Research: IA: Quantifying Plankton Diversity with Taxonomy and Attribute Based Classifiers of Underwater Microscope Images
大数据:合作研究:IA:利用水下显微镜图像的分类和属性分类器量化浮游生物多样性
- 批准号:
1546305 - 财政年份:2016
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
NRI-Small: A Biologically Plausible Architecture for Robotic Vision
NRI-Small:一种生物学上合理的机器人视觉架构
- 批准号:
1208522 - 财政年份:2012
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
Large-vocabulary Semantic Image Processing: Theory and Algorithms
大词汇量语义图像处理:理论与算法
- 批准号:
0830535 - 财政年份:2008
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
RI-Small: Optimal Automated Design of Cascaded Object Detectors
RI-Small:级联物体检测器的优化自动化设计
- 批准号:
0812235 - 财政年份:2008
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
Understanding Video of Crowded Environments
了解拥挤环境的视频
- 批准号:
0534985 - 财政年份:2005
- 资助金额:
$ 71.91万 - 项目类别:
Continuing Grant
CAREER: Weakly Supervised Recognition
职业:弱监督识别
- 批准号:
0448609 - 财政年份:2005
- 资助金额:
$ 71.91万 - 项目类别:
Continuing Grant
相似国自然基金
Immuno-Real Time PCR法精确定量血清MG7抗原及在早期胃癌预警中的价值
- 批准号:30600737
- 批准年份:2006
- 资助金额:22.0 万元
- 项目类别:青年科学基金项目
无色ReAl3(BO3)4(Re=Y,Lu)系列晶体紫外倍频性能与器件研究
- 批准号:60608018
- 批准年份:2006
- 资助金额:28.0 万元
- 项目类别:青年科学基金项目
相似海外基金
An implantable biosensor microsystem for real-time measurement of circulating biomarkers
用于实时测量循环生物标志物的植入式生物传感器微系统
- 批准号:
2901954 - 财政年份:2028
- 资助金额:
$ 71.91万 - 项目类别:
Studentship
HoloSurge: Multimodal 3D Holographic tool and real-time Guidance System with point-of-care diagnostics for surgical planning and interventions on liver and pancreatic cancers
HoloSurge:多模态 3D 全息工具和实时指导系统,具有护理点诊断功能,可用于肝癌和胰腺癌的手术规划和干预
- 批准号:
10103131 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
EU-Funded
CSR: Small: Multi-FPGA System for Real-time Fraud Detection with Large-scale Dynamic Graphs
CSR:小型:利用大规模动态图进行实时欺诈检测的多 FPGA 系统
- 批准号:
2317251 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
CAREER: Real-Time First-Principles Approach to Understanding Many-Body Effects on High Harmonic Generation in Solids
职业:实时第一性原理方法来理解固体高次谐波产生的多体效应
- 批准号:
2337987 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
Continuing Grant
CAREER: Secure Miniaturized Bio-Electronic Sensors for Real-Time In-Body Monitoring
职业:用于实时体内监测的安全微型生物电子传感器
- 批准号:
2338792 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
Continuing Grant
CAREER: Towards Safety-Critical Real-Time Systems with Learning Components
职业:迈向具有学习组件的安全关键实时系统
- 批准号:
2340171 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
Continuing Grant
PZT-hydrogel integrated active non-Hermitian complementary acoustic metamaterials with real time modulations through feedback control circuits
PZT-水凝胶集成有源非厄米互补声学超材料,通过反馈控制电路进行实时调制
- 批准号:
2423820 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant
CAREER: Personalized, wearable robot mobility assistance considering human-robot co-adaptation that incorporates biofeedback, user coaching, and real-time optimization
职业:个性化、可穿戴机器人移动辅助,考虑人机协同适应,结合生物反馈、用户指导和实时优化
- 批准号:
2340519 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
Continuing Grant
CAREER: SHF: Bio-Inspired Microsystems for Energy-Efficient Real-Time Sensing, Decision, and Adaptation
职业:SHF:用于节能实时传感、决策和适应的仿生微系统
- 批准号:
2340799 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
Continuing Grant
I-Corps: Translation Potential of Simultaneous Musculoskeletal Assessment with Real-Time Ultrasound
I-Corps:实时超声同步肌肉骨骼评估的转化潜力
- 批准号:
2413735 - 财政年份:2024
- 资助金额:
$ 71.91万 - 项目类别:
Standard Grant