L2S for CMOS Image Sensor Design and Bridging the Gap

最近，CMOS图像传感器已经成为改变我们生活的关键。此外，这些数码相机产生的图像也是最近人工智能技术发展的主要贡献者。然而，传统的数码相机是为了匹配电脑显示器和纸张等显示器而制造的，而不是为了各种机器学习任务而优化的。尽管许多人工智能算法受到动物大脑的启发，但图像传感器很少从动物眼睛中获得灵感，几乎不进行图像处理。因此，大多数数码相机仅仅是产生数据的输入，信息提取有限。在这个项目中，我们将开发和构建与智能算法共同设计的新型图像传感器架构。为此，一方面，我们将重新审视像素几何和布局，重新设计，为其他L2S项目中的合作者开发的几个图像处理任务提供最佳输入。同时，我们将嵌入接近像素和阵列的早期模拟信号处理，以数学运算符的形式，如导数和卷积，以及典型神经网络的输入阶段。这些将被设计为减少软件中人工智能系统后续计算任务的复杂性。传感器开发人员和人工智能研究人员之间的合作研究将确保在此过程中既不会损失光学效率，也不会损失计算效率。此外，该项目将通过协调两个社区之间的知识交流和联合研究，在智能算法研究人员和多个传感器开发人员之间建立桥梁。

CMOS image sensors have been key to revolutionise our life recently. In addition, images produced by these digital cameras are also a leading contributor to recent growth in AI technologies. However, traditional digital cameras have been built to match displays like computer monitors and paper, rather than being optimised for various machine learning tasks. Despite a number of AI algorithms being inspired by the animal brain, the image sensor seeks little inspiration from animal eye and undertakes almost no image processing. As a result, most digital cameras are mere data producing inputs with limited information extraction. In this project, we will develop and build novel image sensor architectures which are co-designed with intelligent algorithms. To do so, at one hand we will revisit the pixel geometry and layout, redesigning to provide optimal inputs to several image processing tasks developed by our collaborators in other L2S projects. Simultaneously, we will embed early analogue signal processing close to the pixel and the array in form of mathematical operators like derivatives and convolution as well as input stages of typical neural networks. These will be designed to reduce the complexity of the succeeding computational tasks of AI systems in software. The collaborative research between sensor developer and AI researchers will ensure that neither optical nor computation efficiency is lost in this process. Furthermore, this project will build bridge between intelligent algorithm researchers and several sensor developers by coordinating the efforts of knowledge exchange and joint research between the two communities.