Collaborative Research: RI: Medium: Lie group representation learning for vision

建造能够感知、理解环境并在其环境中行动的智能机器，是我们这个时代最大的科学挑战之一。尽管人工智能（AI）最近取得了进展，但实现能够理解物理世界并与之交互的鲁棒自主视觉系统仍然难以捉摸。在数学上，视觉需要理解各种物体形状之间的关系，每个物体都有各种各样的几何和光照变换，从而导致可能的视觉场景的爆炸。该项目旨在通过开发一种基于数学的视觉计算理论来突破这一障碍，该理论将使一类新的神经网络学习算法能够将视觉场景解析为它们的组成对象和转换，从而使计算机能够更好地代表它们周围的世界。这项研究的结果和计算工具将通过课程、研讨会、黑客马拉松和对Geomstats库贡献的开源软件向科学界和公众传播。这个项目的前提是，目前人工智能和计算机视觉的局限性可以通过一个适当的数学框架来解决，即李理论，该理论模拟了视觉世界中自然变换的层次结构。研究人员将通过编码在可学习的g模块（群模块）中的显式李群运算，发展基本信号处理变换的泛化。这些模块通过将图像分解成形状及其底层转换，直接解决视觉中的组合爆炸问题。具体来说，该团队将开发g模块，学习自然图像中包含的变换的群等价表示（目标1），通过仅针对特定变换坍缩群轨道来实现形状的鲁棒表示（目标2），以及通过分解变换和形状的解纠缠（目标3）。模块被组装成层次结构，可以学习转换和形状的复杂表示（目标4）。总之，这些目标提供了一个新的范例，为现有的视觉模型奠定了基础，并为未来深度学习架构的设计提供了一套指导原则，这些架构具有增强的感知和理解世界的能力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The quest to build intelligent machines capable of sensing, understanding and acting in their environment presents one of the great scientific challenges of our time. Despite recent advances in artificial intelligence (AI), the realization of robust, autonomous vision systems that understand and interact with the physical world remains elusive. Mathematically, vision requires understanding the relationships among an immense variety of object shapes, each subject to an immense variety of geometric and lighting transformations, leading to an explosion of possible visual scenes. This project aims to break through this barrier by developing a mathematically grounded computational theory of vision that will enable a new class of neural network learning algorithms to parse visual scenes into their constituent objects and transformations, thereby enabling computers to better represent the world around them. The results and computational tools arising from this research will be disseminated to the scientific community and general public through courses, seminars, hackathons, and open-source software contributed to the Geomstats library.The premise of this project is that the current limitations of AI and computer vision can be addressed with an appropriate mathematical framework, Lie theory, that models the hierarchical structure of natural transformations in the visual world. The investigators will develop generalizations of foundational signal processing transforms through explicit Lie group operations encoded in learnable G-Modules (Group-Modules). These modules directly tackle the combinatoric explosion in vision by factorizing images into shapes and their underlying transformations. Specifically, the team will develop G-modules that learn group-equivariant representations of the transformations contained in natural images (Aim 1), robust representations of shape by collapsing group orbits only with respect to specific transformations (Aim 2), and disentangling of transformation and shape via factorization (Aim 3). The modules are assembled into hierarchical architectures that can learn complex representations of transformations and shapes (Aim 4). Together, these aims provide a new paradigm that grounds existing models of vision and gives a set of guiding principles for the design of future deep learning architectures with enhanced abilities to sense and understand the world.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.