Collaborative Research: RI: Medium: Lie group representation learning for vision

建造能够在环境中感知、理解和行动的智能机器的追求，是我们这个时代的重大科学挑战之一。尽管人工智能(AI)最近取得了进展，但实现理解物理世界并与之交互的强大、自主的视觉系统仍然难以实现。从数学上讲，视觉需要了解各种各样的物体形状之间的关系，每个物体形状都会受到各种各样的几何和光线变换的影响，从而导致可能的视觉场景的爆炸。该项目旨在通过开发一种基于数学的视觉计算理论来突破这一障碍，该理论将使一类新的神经网络学习算法能够将视觉场景解析为其组成对象和变换，从而使计算机能够更好地表示它们周围的世界。这项研究的结果和计算工具将通过课程、研讨会、黑客马拉松和为Geomstats图书馆提供的开源软件向科学界和普通公众传播。该项目的前提是，可以通过适当的数学框架来解决当前人工智能和计算机视觉的局限性，谎言理论对视觉世界中自然变换的层次结构进行建模。研究人员将通过在可学习G-模(群-模)中编码的显式李群运算来开发基本信号处理变换的一般化。这些模块通过将图像分解成形状及其基本变换，直接处理视觉中的组合爆炸。具体地说，该团队将开发G-模块，学习自然图像中包含的变换的群等变表示(目标1)，通过仅相对于特定变换折叠群轨道来稳健表示形状(目标2)，以及通过因式分解解开变换和形状的纠缠(目标3)。这些模块被组装成可以学习转换和形状的复杂表示的分层体系结构(目标4)。总而言之，这些目标提供了一个新的范式，以现有的愿景模型为基础，并为未来具有增强感知和理解世界能力的深度学习架构的设计提供了一套指导原则。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The quest to build intelligent machines capable of sensing, understanding and acting in their environment presents one of the great scientific challenges of our time. Despite recent advances in artificial intelligence (AI), the realization of robust, autonomous vision systems that understand and interact with the physical world remains elusive. Mathematically, vision requires understanding the relationships among an immense variety of object shapes, each subject to an immense variety of geometric and lighting transformations, leading to an explosion of possible visual scenes. This project aims to break through this barrier by developing a mathematically grounded computational theory of vision that will enable a new class of neural network learning algorithms to parse visual scenes into their constituent objects and transformations, thereby enabling computers to better represent the world around them. The results and computational tools arising from this research will be disseminated to the scientific community and general public through courses, seminars, hackathons, and open-source software contributed to the Geomstats library.The premise of this project is that the current limitations of AI and computer vision can be addressed with an appropriate mathematical framework, Lie theory, that models the hierarchical structure of natural transformations in the visual world. The investigators will develop generalizations of foundational signal processing transforms through explicit Lie group operations encoded in learnable G-Modules (Group-Modules). These modules directly tackle the combinatoric explosion in vision by factorizing images into shapes and their underlying transformations. Specifically, the team will develop G-modules that learn group-equivariant representations of the transformations contained in natural images (Aim 1), robust representations of shape by collapsing group orbits only with respect to specific transformations (Aim 2), and disentangling of transformation and shape via factorization (Aim 3). The modules are assembled into hierarchical architectures that can learn complex representations of transformations and shapes (Aim 4). Together, these aims provide a new paradigm that grounds existing models of vision and gives a set of guiding principles for the design of future deep learning architectures with enhanced abilities to sense and understand the world.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.