CAREER: Vision Systems for an Evolving World

当过渡到一个黑暗的房间时，大多数人最初都很难看清，但很快就能适应并在新的环境中看清。事实上，即使世界的外观以许多不同的方式变化，人们仍在继续观察和理解他们周围的世界。相比之下，如果世界发生变化，我们的计算机视觉系统理解世界的能力有限。想象一下，如果一个人第一次在黄昏或雪地里开车，他就认不出路来了。毫无疑问，驾驶员培训需要在一天中的任何时候、在不同的天气条件下、在不同的城市等等，在允许新驾驶员驾驶之前，坐在副驾驶座位上许多小时。该项目旨在研究和建立新的模型、算法和成功措施，使下一代视觉识别系统能够适应不断发展的视觉世界。该项目将把研究与K-12学生的教育和推广结合起来。该项目通过一种新的集成方法倡导弹性视觉系统，该方法在可用视觉域的泛化和给定新域数据的快速适应之间迭代。先前的方法针对独立的标准进行优化，要么跨多个领域的泛化，要么适应新的目标领域，这限制了随着时间的推移，创建可以在更多领域运行的视觉系统的更大目标的进展。此外，现有解决方案的适应速度很慢，在进行更新之前依赖于大量的新观察结果。该项目将致力于：1)促进未来适应的多领域泛化的模型设计和学习方法。2)变革性视觉域自适应算法，该算法能够使用有限的目标观测值快速适应目标域，而无需访问大量辅助数据源，从而降低计算需求。3)随着时间的推移，使视觉系统能够扩展其成功运行的域集的算法。最后，本项目将引入一个基准和新的评估协议来衡量视觉识别模型对变化域的弹性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

When transitioning into a dark room most people initially struggle to see, but are quickly able to adjust and see in the new setting. In fact, people continue to see and understand the world around them even as the appearance of the world changes in many different ways. In contrast, our computer vision systems have limited ability to understand the world if it changes. Imagine if the first time one drove at twilight or in the snow one could no longer recognize the road. No doubt, driver’s training would require many hours of sitting in the passenger seat at all times of day, within different weather conditions, across different cities, and so on before allowing a new driver behind the wheel. This project aims to study and build new models, algorithms, and measures of success enabling the next generation of visual recognition systems to be resilient to an evolving visual world. The project will integrate research with education and outreach to K-12 students.This project advocates for resilient vision systems through a new integrated approach which iterates between generalizing across available visual domains and rapidly adapting given new domain data. Prior approaches optimize for independent criteria, either generalization across multiple domains, or adaptation to a new target domain, which limits advancement towards the larger goal of creating vision systems that can operate in more domains over time. Further, existing solutions are slow to adapt, relying on substantial new observations before updates can be made. The project will work on: 1) Model design and learning approaches for multi-domain generalization that facilitates future adaptation. 2) Transformative visual domain adaptation algorithms that are capable of rapidly adapting to a target domain using limited target observations and without accessing a large auxiliary source of data, reducing compute demands. 3) Algorithms that enable vision systems to expand the set of domains they can successfully operate in over time. Finally, this project will introduce a benchmark and new evaluation protocols to measure the resilience of visual recognition models to changing domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.