CAREER: Visual Learning in an Open and Continual World

职业:开放和持续世界中的视觉学习

基本信息

  • 批准号:
    2239292
  • 负责人:
  • 金额:
    $ 53.51万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Continuing Grant
  • 财政年份:
    2023
  • 资助国家:
    美国
  • 起止时间:
    2023-02-01 至 2028-01-31
  • 项目状态:
    未结题

项目摘要

The field of computer vision has seen significant progress in the past decade: These models are now able to efficiently process complex images and automatically extract information, such as detecting what type of objects exist in the image and where they are located. However, current methods require a pre-specified list of object categories that are in the images. This requirement that is unrealistic when systems are deployed in real-world contexts, such as on self-driving cars or large photo collections. If new types of objects appear, current systems will need to have humans identify the new objects and annotate the images and then retrain the computer vision model through a process that takes significant computational resources. Unlike humans, the system cannot automatically understand when new types of objects are in the images, how they relate to objects that the system already knows about, and how to continually update its knowledge given little to no human annotation. This project therefore seeks to enable a computer vision system that can continuously and automatically detect and discover new categories, as well as update its model, with little to no human annotation. Such a capability would have implications in a range of applications, including personalized analysis of photo collections, home robotics, self-driving cars, and medical imaging, where novel unknown objects often lead to misleading or incorrect object detection. The project will address this through a range of research innovations as well as through several outreach activities, including democratizing AI education by working with educators from K-12 and up to teach our open-source course materials. Towards this end, the goal of this project is to create a framework for an open-world and continual learning system that develops principled methods for naturally understanding and handling different types of distribution shifts, as well as incrementally discovering and learning new categories as they appear in unlabeled data, and placing them within a rich semantic hierarchical structure. This will be accomplished by first detecting different types of distribution shift that can occur (e.g., changes in appearance due to weather or existence of entirely new objects) and developing principled out-of-distribution detection and calibration methods to disentangle them. These methods will be used to understand how they affect the model's predictions. Subsequently, rather than just detecting whether new categories exist and throwing the resulting data out, this fine-grained understanding of distribution shift will support incrementally updating the model in response. This will be done by developing methods to build long-term representations and classifiers that discover new categories and place them within a rich hierarchical semantic structure. Finally, semi-supervised continual learning will be leveraged to incrementally refine the representations and automatically learn classification and detection models, using a mixture of labeled and unlabeled data appearing at different times, while avoiding catastrophic forgetting.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
在过去的十年里,计算机视觉领域取得了长足的进步:这些模型现在能够高效地处理复杂的图像并自动提取信息,例如检测图像中存在的对象类型及其位置。然而,当前的方法需要图像中的对象类别的预先指定的列表。当系统部署在真实世界的环境中时,例如在自动驾驶汽车或大型照片收集上,这一要求是不现实的。如果出现新类型的物体,目前的系统将需要人类识别新物体并对图像进行注释,然后通过一个需要大量计算资源的过程重新训练计算机视觉模型。与人类不同,该系统不能自动理解图像中何时出现新类型的对象,它们如何与系统已知的对象相关,以及如何在几乎没有人类注释的情况下不断更新其知识。因此,该项目试图使计算机视觉系统能够连续和自动地检测和发现新的类别,并更新其模型,几乎不需要人工注释。这种能力将在一系列应用中产生影响,包括照片收藏的个性化分析、家用机器人、自动驾驶汽车和医学成像,在这些应用中,新颖的未知对象往往会导致误导或错误的对象检测。该项目将通过一系列研究创新以及几项外联活动来解决这一问题,包括通过与K-12及以上的教育工作者合作教授我们的开源课程材料来实现人工智能教育的民主化。为此,该项目的目标是建立一个开放世界和持续学习系统的框架,该系统开发出原则性的方法,以便自然地理解和处理不同类型的分配变化,并逐步发现和学习出现在未标记数据中的新类别,并将其置于丰富的语义等级结构中。这将通过首先检测可能发生的不同类型的分布偏移(例如,由于天气或全新物体的存在而引起的外观变化)并开发原则性的分布外检测和校准方法来解决它们来实现。这些方法将被用来理解它们如何影响模型的预测。随后,不只是检测是否存在新的类别并丢弃结果数据,这种对分布变化的细粒度理解将支持作为响应的增量更新模型。这将通过开发方法来构建长期表示和分类器来实现,这些方法发现新的类别并将它们置于丰富的分层语义结构中。最后,半监督的持续学习将被利用来增量地改进表示法,并自动学习分类和检测模型,使用在不同时间出现的标记和未标记数据的混合,同时避免灾难性遗忘。该奖项反映了NSF的法定使命,并通过使用基金会的智力优势和更广泛的影响审查标准进行评估,被认为值得支持。

项目成果

期刊论文数量(3)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
HePCo: Data-Free Heterogeneous Prompt Consolidation for Continual Federated Learning
  • DOI:
    10.48550/arxiv.2306.09970
  • 发表时间:
    2023-06
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Shaunak Halbe;James Smith;Junjiao Tian;Z. Kira
  • 通讯作者:
    Shaunak Halbe;James Smith;Junjiao Tian;Z. Kira
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning
  • DOI:
    10.1109/cvpr52729.2023.01146
  • 发表时间:
    2022-11
  • 期刊:
  • 影响因子:
    0
  • 作者:
    James Smith;Leonid Karlinsky;V. Gutta;Paola Cascante-Bonilla;Donghyun Kim-;Assaf Arbelle;Rameswar Panda;R. Feris;Z. Kira
  • 通讯作者:
    James Smith;Leonid Karlinsky;V. Gutta;Paola Cascante-Bonilla;Donghyun Kim-;Assaf Arbelle;Rameswar Panda;R. Feris;Z. Kira
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Zsolt Kira其他文献

Biological underpinnings for lifelong learning machines
终身学习机器的生物学基础
  • DOI:
    10.1038/s42256-022-00452-0
  • 发表时间:
    2022-03-23
  • 期刊:
  • 影响因子:
    23.900
  • 作者:
    Dhireesha Kudithipudi;Mario Aguilar-Simon;Jonathan Babb;Maxim Bazhenov;Douglas Blackiston;Josh Bongard;Andrew P. Brna;Suraj Chakravarthi Raja;Nick Cheney;Jeff Clune;Anurag Daram;Stefano Fusi;Peter Helfer;Leslie Kay;Nicholas Ketz;Zsolt Kira;Soheil Kolouri;Jeffrey L. Krichmar;Sam Kriegman;Michael Levin;Sandeep Madireddy;Santosh Manicka;Ali Marjaninejad;Bruce McNaughton;Risto Miikkulainen;Zaneta Navratilova;Tej Pandit;Alice Parker;Praveen K. Pilly;Sebastian Risi;Terrence J. Sejnowski;Andrea Soltoggio;Nicholas Soures;Andreas S. Tolias;Darío Urbina-Meléndez;Francisco J. Valero-Cuevas;Gido M. van de Ven;Joshua T. Vogelstein;Felix Wang;Ron Weiss;Angel Yanguas-Gil;Xinyun Zou;Hava Siegelmann
  • 通讯作者:
    Hava Siegelmann

Zsolt Kira的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Zsolt Kira', 18)}}的其他基金

NRI: Large-Scale Collaborative Semantic Mapping using 3D Structure from Motion
NRI:使用 Motion 的 3D 结构进行大规模协作语义映射
  • 批准号:
    1426998
  • 财政年份:
    2014
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Continuing Grant

相似国自然基金

基于多幅图象的Visual Hull重构及表面属性建模算法研究
  • 批准号:
    60373031
  • 批准年份:
    2003
  • 资助金额:
    23.0 万元
  • 项目类别:
    面上项目

相似海外基金

IUSE: Conservation Principles, Illustrated: Analyzing the Impact of Informal Visual Learning Tools on Educational Engineering Through Comics
IUSE:保护原则,图解:通过漫画分析非正式视觉学习工具对教育工程的影响
  • 批准号:
    2235827
  • 财政年份:
    2024
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Standard Grant
I-Corps: Real-time visual feedback for speech learning
I-Corps:语音学习的实时视觉反馈
  • 批准号:
    2408991
  • 财政年份:
    2024
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Standard Grant
Using Visual Computing to Deepen Mathematical Learning in Preservice K-8 Teacher Education
使用视觉计算深化职前 K-8 教师教育中的数学学习
  • 批准号:
    2337247
  • 财政年份:
    2024
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Standard Grant
Collaborative Research: Using Machine Learning to Improve Visual Problem-Solving in Chemistry Education
协作研究:利用机器学习提高化学教育中的视觉问题解决能力
  • 批准号:
    2235790
  • 财政年份:
    2023
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Standard Grant
Cloud-Based Machine Learning and Biomarker Visual Analytics for Salivary Proteomics
基于云的机器学习和唾液蛋白质组生物标志物可视化分析
  • 批准号:
    10827649
  • 财政年份:
    2023
  • 资助金额:
    $ 53.51万
  • 项目类别:
NSF-BSF: Reactivation and sleep in visual and motor skill acquisition: learning beyond training
NSF-BSF:视觉和运动技能习得中的重新激活和睡眠:训练之外的学习
  • 批准号:
    2241417
  • 财政年份:
    2023
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Standard Grant
Rotation 1: Biological plausible models of visual perceptual learning
旋转 1:视觉感知学习的生物学合理模型
  • 批准号:
    2887737
  • 财政年份:
    2023
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Studentship
Toward describing fine-grained details in computer vision through visual discrimination learning
通过视觉辨别学习来描述计算机视觉中的细粒度细节
  • 批准号:
    23K16945
  • 财政年份:
    2023
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Grant-in-Aid for Early-Career Scientists
CAREER: Investigating Curiosity-Driven Visual Processing during Science Learning
职业:研究科学学习过程中好奇心驱动的视觉处理
  • 批准号:
    2239591
  • 财政年份:
    2023
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Continuing Grant
CRCNS Research Proposal: Learning by Looking: Modeling visual system representation formation via foveated sensing in a 3-D world
CRCNS 研究提案:通过观察学习:通过 3D 世界中的注视点感知对视觉系统表征形成进行建模
  • 批准号:
    2309041
  • 财政年份:
    2023
  • 资助金额:
    $ 53.51万
  • 项目类别:
    Continuing Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了