Integrating Federated Split Neural Network with Artificial Stereoscopic Compound Eyes for Optical Flow Sensing in 3D Space with Precision

该项目旨在开发一种创新的图像传感器，其灵感来自节肢动物的眼睛，具有宽视野，高速操作和高效的目标跟踪。这些智能传感器可以大大扩展视野，提高响应速度，并以节能方式运行。这一进步的核心是光电二极管与人工突触的融合以及几何形状的模仿，这一配对反映了昆虫眼睛内神经网络的生物过程。这种配置不仅有利于视觉信息的快速处理，而且还减少了这样做所需的能量，标志着从传统的平面成像系统向前迈出了重要的一步。与传统系统不同，传感器阵列将采用半球形设计，灵感来自大自然自己的解决方案，以广角和有效的视觉。这种几何优化对于以更高的精度和深度捕获视觉场景中的运动模式至关重要，使得该技术对于需要精确运动检测和空间感知的应用非常宝贵。为了处理这些传感器捕获的高维数据，该项目进一步引入了一个专门的神经网络架构。这种方法在传感器和中央处理器之间划分数据处理任务，确保快速，高效和精确的分析。这种配置特别适合于快速决策至关重要的动态环境。通过模拟自然界中复杂的视觉系统，它提供了一个技术和生物融合的未来，提供了有影响力的解决方案。该项目的目标是开发和集成一种新型人工复眼系统，该系统具有先进的原位物体跟踪和深度感知能力。该项目的目标是开发和集成一种新型人工复眼系统，该系统可以在机器人、自动驾驶等领域提高系统的效率和能力，从而推动科学技术的进步，造福社会。该系统利用了与人工突触设备集成的光电二极管的独特优势，体现了类似于生物突触功能的即时数据处理的计算层。硬件设计的灵感来自节肢动物眼睛的半球形结构，能够实现宽视场和快速图像采集。为了有效地处理高维视觉数据的挑战，将采用通信，存储和节能的联合分裂学习框架。该框架通过在传感器和中央处理单元之间分配计算任务来优化数据处理，从而显著增强系统的实时对象跟踪能力。通过将先进的硬件与软件算法相结合，该项目旨在创建一个系统，不仅可以推进对生物启发成像的科学理解，还可以为现实世界的应用提供实用的解决方案。该项目的成功实施有望为光学传感技术树立新的标准，为计算机视觉和神经形态计算领域做出重大贡献。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project is aiming to develop an innovative image sensor inspired by arthropod eyes, featuring a wide field of view, high-speed operation, and efficient object tracking. These smart sensors can dramatically expand the field of view, enhance response speeds, and operate with energy efficiency. Central to this advancement is the fusion of photodiodes with artificial synapses and mimicry of geometrical shape, a pairing that mirrors the biological processes of neural networks within the insect eye. This configuration not only facilitates the rapid processing of visual information but also reduces the energy required to do so, marking a significant step forward from traditional planar imaging systems. Unlike conventional systems, the sensor arrays will adopt hemispherical designs, inspired by nature's own solution to wide-angle and efficient vision. This geometric optimization is crucial for capturing the patterns of movement across a visual scene with enhanced accuracy and depth, making the technology invaluable for applications that require precise motion detection and spatial awareness. To process the high-dimensional data captured by these sensors, the project further introduces a specialized neural network architecture. This approach divides data processing tasks between the sensors and a central processing unit, ensuring swift, efficient, and precise analysis. Such a configuration is particularly suited for dynamic environments where rapid decision-making is essential. By emulating the intricate vision systems found in nature, it offers a glimpse into a future where technology and biology converge, providing impactful solutions. The potential applications of this work span across various sectors, promising to enhance the efficiency, and capabilities of systems in robotics, autonomous driving, and beyond, thereby advancing the progress of science and technology for the benefit of society.The goal of this project is to develop and integrate a novel artificial compound eye system capable of advanced in-situ object tracking and depth perception. This system leverages the unique advantages of a photodiode integrated with an artificial synaptic device, embodying a computational layer for immediate data processing akin to biological synaptic functions. The hardware design, inspired by the hemispherical structure of arthropod eyes, enables a wide field of view and rapid image acquisition. To address the challenges of processing high-dimensional visual data efficiently, a communication, storage, and energy-efficient federated split learning framework will be employed. This framework optimizes data processing by distributing computational tasks between the sensor and a centralized processing unit, significantly enhancing the system's real-time object tracking capabilities. By integrating advanced hardware with software algorithms, this project aims to create a system that not only advances the scientific understanding of bio-inspired imaging but also offers practical solutions for real-world applications. The successful execution of this project is expected to set a new standard in optical sensing technology, contributing significantly to the fields of computer vision and neuromorphic computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.