CAREER: Vision Systems for an Evolving World

当过渡到黑暗的房间时，大多数人一开始很难看到东西，但很快就能在新的环境中调整和看到东西。事实上，即使世界的面貌发生了许多不同的变化，人们仍在继续看到和理解他们周围的世界。相比之下，如果世界发生变化，我们的计算机视觉系统理解世界的能力有限。想象一下，如果一个人第一次在黄昏或雪地里开车时，他再也认不出路来了。毫无疑问，司机培训需要在一天中的任何时间、在不同的天气条件下、在不同的城市等情况下坐在乘客座位上数小时，然后才能让新司机开车。该项目旨在研究和构建新的模型、算法和成功措施，使下一代视觉识别系统能够适应不断发展的视觉世界。该项目将把研究与教育和对K-12学生的推广相结合。该项目通过一种新的综合方法倡导弹性视觉系统，该方法在可用视觉领域进行概括和快速适应给定的新领域数据之间进行迭代。以前的方法针对独立的标准进行了优化，要么是跨多个领域的泛化，要么是适应新的目标领域，这限制了随着时间的推移创建可以在更多领域运行的视觉系统这一更大目标的进展。此外，现有的解决方案适应速度很慢，在进行更新之前依赖于大量的新观察。该项目将在以下方面开展工作：1)促进未来适应的多领域推广的模式设计和学习方法。2)变换视域自适应算法，其能够使用有限的目标观测快速地自适应目标域，而无需访问大量辅助数据源，从而减少计算需求。3)算法，使视觉系统能够随着时间的推移扩展它们可以成功操作的域的集合。最后，这个项目将引入一个基准和新的评估协议来衡量视觉识别模型对变化领域的弹性。这个奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

When transitioning into a dark room most people initially struggle to see, but are quickly able to adjust and see in the new setting. In fact, people continue to see and understand the world around them even as the appearance of the world changes in many different ways. In contrast, our computer vision systems have limited ability to understand the world if it changes. Imagine if the first time one drove at twilight or in the snow one could no longer recognize the road. No doubt, driver’s training would require many hours of sitting in the passenger seat at all times of day, within different weather conditions, across different cities, and so on before allowing a new driver behind the wheel. This project aims to study and build new models, algorithms, and measures of success enabling the next generation of visual recognition systems to be resilient to an evolving visual world. The project will integrate research with education and outreach to K-12 students.This project advocates for resilient vision systems through a new integrated approach which iterates between generalizing across available visual domains and rapidly adapting given new domain data. Prior approaches optimize for independent criteria, either generalization across multiple domains, or adaptation to a new target domain, which limits advancement towards the larger goal of creating vision systems that can operate in more domains over time. Further, existing solutions are slow to adapt, relying on substantial new observations before updates can be made. The project will work on: 1) Model design and learning approaches for multi-domain generalization that facilitates future adaptation. 2) Transformative visual domain adaptation algorithms that are capable of rapidly adapting to a target domain using limited target observations and without accessing a large auxiliary source of data, reducing compute demands. 3) Algorithms that enable vision systems to expand the set of domains they can successfully operate in over time. Finally, this project will introduce a benchmark and new evaluation protocols to measure the resilience of visual recognition models to changing domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.