CAREER: Understanding visual learning with self-supervised neural network models

当今人工智能的一个核心问题是，机器学习算法通常需要使用大量手工数据进行监督训练。因此，这些算法的范围在很大程度上局限于资金充足的组织可以构建大量的、专业注释的、通常专有的、标记的数据集的领域。相比之下，真实的生物系统，如人类婴儿，学习效率要高得多，将少量的明确监督与强大的（但尚未完全理解的）自我监督机制相结合。该提案旨在构建受生物启发的通用自我监督系统，该系统可以在不需要数百万标记示例的情况下进行学习。实现这一目标的基本策略将是开发和完善新兴的无监督深度学习领域的技术，其中神经网络训练自己捕捉感官环境中存在的微妙统计模式。这些网络将被增强，在丰富的交互式物理领域中作为代理人运行，在那里它们将寻找具有挑战性但最终可解决的自我监督“目标”，这将教会它们灵活地表示和响应环境。如果成功，这样的系统将有能力使用在物理世界中无处不在的大量未标记数据。该提案还试图将这些算法思想用作真实的生物系统中学习的定量模型的假设。使用最近开发的计算神经科学技术，神经网络将被比较使用广泛的实验范式收集的神经和行为数据。然后，将确定哪些自监督神经网络学习模型最好地捕获经验数据-同样重要的是，实验和计算模型之间最明显的不匹配在哪里。量化这些模型数据比较将反过来允许反馈，以构建更好的神经网络算法。这项工作的最终目标是在实验观察和计算算法开发之间建立一个紧密的循环，加速人工智能和神经科学的进步。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A central problem in artificial intelligence today is that machine learning algorithms often require supervised training with huge amounts of hand-curated data. As a result, such algorithms are largely limited in scope to domains where well-funded organizations can build massive, expertly-annotated, and typically proprietary, labelled datasets. In contrast, real biological systems such as human infants learn much more efficiently, combining a small amount of explicit supervision with powerful -- but not fully understood -- mechanisms of self-supervision. This proposal seeks to build biologically-inspired general-purpose self-supervised systems that can learn without needing to be spoon-fed millions of labeled examples. The basic strategy to achieve this goal will be to develop and refine techniques in the emerging field of unsupervised deep learning, in which neural networks train themselves to capture the subtle statistical patterns present in their sensory surroundings. These networks will be augmented to operate as agents in a rich interactive physical domain, where they will seek out challenging but ultimately solvable self-supervised "goals" that will teach them to flexibly represent and respond to their environment. If successful, such systems will have the ability to use the wealth of unlabeled data that is ubiquitously available in the physical world. The proposal also seeks to use these algorithmic ideas as hypotheses for quantitative models of learning in real biological systems. Using recently developed techniques from computational neuroscience, the neural networks will be compared to neural and behavioral data collected using a wide spectrum of experimental paradigms. It will then be determined which self-supervised neural network learning models best capture the empirical data -- and equally importantly, where the most glaring mismatches between experiment and computational models lie. Quantifying these model-data comparisons will in turn allow for feedback to build better neural network algorithms. The ultimate goal of this work is to set up a tight loop between experimental observation and computational algorithm development, accelerating progress both in artificial intelligence and neuroscience.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.