FuSe: Co-designed Systems for In-sensor Processing with Sustainable Nanomaterials (COSMIC)

该提案旨在开发人工智能（AI）和机器学习（ML）的新技术。将使用环境上可持续的材料，开发一个信息处理系统，特别是图像处理系统。目前的AI模型，如OpenAI的ChatGPT，需要大量的电力和资源来运行。越来越多的公司急于创建他们的ChatGPT克隆，这是不可持续的，只会加剧气候变化。提出了用于视觉传感的专用硬件，其占用空间更小，更节能，可以在笔记本电脑等传统设备上运行AI模型，而无需依赖云服务器。人工视觉传感器可以通过实现超越人眼能力的视觉来改变未来，从而影响现代生活的整个景观。通过检测运动的变化，它将为未来的“智能”城市带来新一代的自动驾驶汽车。拟议的视觉传感器将能够改善对行业过程的监控，并提高对运动员身体运动的理解，所有这些都具有低能耗和小面积。这项工作的科学发现和进步所带来的未来产品将增强半导体行业。例如，完全可回收的电子纳米材料的改进合成工艺将补充该行业的增长，创造新的就业机会。半导体劳动力的发展将得到大力强调，并通过这项赠款，通过该项目培训的学生将有能力加入众多的工业努力伴随着CHIPS美国法案和其他举措。当地大学生将接触到的发展，并培训为半导体行业的劳动力。高中教师将接触到半导体领域的前沿研究，向学生传达该领域的热情。传感器内计算的材料-设备-系统协同设计，像素阵列与交叉阵列异质集成，形成用于图像处理的卷积神经网络（CNN），特别强调可持续制造，由Co-designed Systems for In-sensor Processing with Sustainable Nanomaterials（COSMIC）团队提出。该团队将寻求共同设计和异构集成，以实现生物启发的感知到行动，从而使用机器学习架构优化功耗、性能和硬件占用空间。这些目标的追求将由神经形态设备（罗伊），可持续电子（富兰克林）和神经形态电路和算法（李）的纳米材料专家承担。目标是：A.开发具有每个像素的最高可能填充因子的像素内计算电路; B.将像素阵列与基于纵横的存储器内计算电路集成，以实现用于图像分类和分割的CNN;以及C.使用可回收材料开发整个过程，以减少电子废物。对于像素内计算，将设计和优化使用二维材料作为沟道和栅电极的具有多波长传感能力的新型光电突触器件。像素内计算电路形成CNN的第一层。CNN的后续层将通过忆阻突触的交叉阵列来实现。材料-设备协同设计将确保突触设备的最佳特性用于像素内计算和交叉开关实现，而设备-系统协同设计将减轻剩余的非理想性，同时通过混合模拟/数字计算方案最大化性能。纳米材料的外围电路将根据交叉电路的器件性能和精度要求进行共同设计。将研究所有电路的可回收性，并确保可持续制造。最后，基于光电突触的像素阵列将与交叉阵列和外围电路异质地集成，并且该硬件用于图像处理任务的性能，例如分类和分割，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。基金会的使命是履行其法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The proposal aims at developing a new technology for artificial intelligence (AI) and machine learning (ML). Using environmentally sustainable materials, a system for processing information – specifically for image processing – will be developed. The current AI models, such as OpenAI’s ChatGPT, require a massive amount of power and resources to run. More companies are rushing to create their ChatGPT clones, which is not sustainable and will only exacerbate climate change. Specialized hardware for vision sensing that is smaller in footprint and more energy-efficient to run AI models on conventional devices like laptops, without relying on cloud servers, is proposed. Artificial vision sensors can transform the future by enabling vision beyond the human eye’s capabilities, leading to impact across the entire landscape of modern life. By detecting changes in motion, it will lead to a new generation of self-driving cars for future “smarter” cities. The proposed vision sensors will enable improved monitoring of processes in the industry, and improved understanding of body movements of athletes, all with low energy consumption and on a small area. Future products enabled by the scientific discoveries and advancements of this work will augment the semiconductor industry. For instance, improved synthesis processes for electronic nanomaterials that are fully recyclable will supplement the industry’s growth, creating new job opportunities. Semiconductor workforce development will be strongly emphasized and enabled by this grant, and students trained through this project will be equipped to join the multitudes of industrial efforts accompanied by the CHIPS for America act and other initiatives. Local college students will be exposed to the developments and trained as a workforce for the semiconductor industry. High school teachers will be exposed to cutting-edge research in semiconductors to convey the enthusiasm of the field to their students.A materials-devices-system co-design for in-sensor computing with pixel arrays heterogeneously integrated with crossbar arrays to form a convolutional neural network (CNN) for image processing, with a special emphasis on sustainable manufacturing, is proposed by the Co-designed Systems for In-sensor Processing with Sustainable Nanomaterials (COSMIC) team. The team will pursue co-design and heterogeneous integration for bio-inspired sensing-to-action to optimize power consumption, performance, and hardware footprint using machine learning architectures. Pursuit of these goals will be undertaken by experts in neuromorphic devices (Roy), nanomaterials for sustainable electronics (Franklin), and neuromorphic circuits and algorithms (Li). The objectives are to: A. Develop in-pixel computing circuits with the highest possible fill factor of each pixel; B. Integrate the pixel arrays with a crossbar-based in-memory computing circuitry to realize a CNN for image classification and segmentation; and C. Develop the entire process with recyclable materials to reduce electronic waste. For in-pixel computing, novel optoelectronic synaptic devices using two-dimensional materials as channel and gate electrode, with multi-wavelength sensing capabilities will be designed and optimized. The in-pixel computing circuit forms the first layer of the CNN. Subsequent layers of the CNN will be realized with crossbar arrays of memristive synapses. Materials-device co-design will ensure the best characteristics of the synaptic devices for in-pixel computing and crossbar implementation, while device-system co-design will mitigate the remaining non-idealities while maximizing performance through a hybrid analog/digital computing scheme. Peripheral circuits with nanomaterials will be co-designed based on device performance and precision requirements of the crossbar circuits. The recyclability of all circuits will be studied and ensured for sustainable manufacturing. Finally, the optoelectronic synapse-based pixel arrays will be heterogeneously integrated with the crossbar arrays and peripheral circuitry, and the performance of this hardware for image processing tasks, such as classification and segmentation, will be evaluated.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteriaThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.