RI: Small: Sparse Predictive Coding for Energy Efficient Visual Navigation in Dynamic Environments

该项目开发了高效的机器视觉算法，灵感来自灵长类动物视觉系统的结构和能量效率，用于运动处理。在一个丰富而混乱的自然环境中导航，同时观察者和场景中的物体都在移动，这是机器视觉中的一个难题，特别是在功率限制下的实时处理。然而，人类和其他动物可以轻松地完成这些任务。神经系统受到严格的代谢限制，这导致了对重要环境特征的高效表征，比如观察者的方向、物体的深度和物体的运动。此外，这些高效的机器视觉算法可以应用于机器人、物联网和边缘处理。这些算法可以应用于广泛的应用，包括增强现实、辅助机器人、自动驾驶汽车和物联网（IoT），因此，通过创建可以在偏远地区长期自主运行的应用程序，它们可以产生变革性的经济和社会影响。受神经系统对构成动态场景的视觉特征进行有效编码和适当响应的能力的启发，该算法使用稀疏预测编码技术来处理来自相机的数据流。由于算法可以在尖峰神经网络中实现，在尖峰神经网络中，人工神经元只在事件发生时发送信号，因此它们可以在低功耗的神经形态系统上高效运行；支持这种表示的计算机。该系统采用了一种受大脑启发的结构，即来自额叶皮层和顶叶皮层的自上而下信号预测物体未来的位置，从而可以更好地跟踪物体，并克服物体隐藏在视线之外时的困难。这些表示是稀疏和简化的，从而导致高效的处理，更少的计算，从而降低功耗。总之，机器视觉算法：(1)增加我们对大脑如何编码世界中的行为相关信号的理解，(2)导致对大数据流的计算效率处理，以及(3)为广泛的嵌入式应用实现节能处理。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project develops efficient machine vision algorithms inspired by the architecture and energetic efficiency of the primate visual system for motion processing. Navigating through a rich cluttered natural environment, while both the observer and the objects in the scene are moving, is a difficult problem in machine vision, particularly for real-time processing under power constraints. However, humans and other animals perform these tasks with ease. The nervous system is under tight metabolic constraints and this leads to incredibly efficient representations of important environmental features, such as the observer's heading, the depth of objects, and the motion of objects. In addition, these efficient machine vision algorithms can be applied to robotics, the IoT, and edge processing. The algorithms can be applied to a wide range of applications, including augmented reality, assistive robotics, autonomous vehicles, and the Internet of Things (IoT) Thus, they could have a transformative economic and societal impact by creating applications that can operate autonomously over long periods in remote locations.Inspired by ability of the nervous system to efficiently encode and appropriately respond to the visual features that make up a dynamic scene, the algorithm uses sparse predictive coding techniques to process data streams from cameras. Because the algorithms can be realized in spiking neural networks, where the artificial neurons only send signals when an event occurs, they can run efficiently on low powered neuromorphic systems; computers that support such representations. By employing an architecture inspired by the brain, where op-down signals from the frontal cortex and parietal cortex predict where objects will be in the future, the system will have better object tracking and overcome difficulties when objects become hidden from view. These representations are sparse and reduced, leading to energy efficient processing, less computation, and thus low power consumption. In summary, the machine vision algorithms: (1) increase our understanding of how the brain encodes behaviorally relevant signals in the world, (2) lead to computationally efficient handling of large data streams, and (3) realize power efficient processing for a wide range of embedded applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.