FuSe: Co-designed Systems for In-sensor Processing with Sustainable Nanomaterials (COSMIC)

该提案旨在开发人工智能（AI）和机器学习（ML）的新技术。使用环境上可持续的材料，将开发一个信息处理系统，特别是图像处理系统。目前的人工智能模型，如OpenAI的ChatGPT，需要大量的电力和资源来运行。越来越多的公司正在争先恐后地创建他们的ChatGPT克隆产品，这是不可持续的，只会加剧气候变化。此外，还提出了在笔记本电脑等传统设备上运行人工智能模型的专用硬件，该硬件占地面积更小，能效更高，无需依赖云服务器。人工视觉传感器可以通过实现超越人眼能力的视觉来改变未来，从而影响整个现代生活。通过检测运动的变化，它将为未来的“智能”城市带来新一代的自动驾驶汽车。所提出的视觉传感器将改进对工业过程的监控，提高对运动员身体运动的理解，所有这些都是低能耗和小面积的。这项工作的科学发现和进步所带来的未来产品将扩大半导体行业。例如，完全可回收的电子纳米材料合成工艺的改进将补充该行业的增长，创造新的就业机会。这项拨款将大力强调和促进半导体劳动力的发展，通过该项目培训的学生将有能力加入伴随着美国芯片法案和其他倡议的众多工业努力。本地大学生将接触到半导体行业的发展，并接受培训，成为半导体行业的劳动力。为了向学生们传达半导体领域的热情，高中教师们将亲自接触半导体领域的尖端研究成果。可持续纳米材料协同设计系统（COSMIC）团队提出了一种用于传感器内计算的材料-设备-系统协同设计，将像素阵列与交叉棒阵列异质集成，形成用于图像处理的卷积神经网络（CNN），特别强调可持续制造。该团队将追求协同设计和异构集成，以实现生物感应到行动，利用机器学习架构优化功耗、性能和硬件足迹。这些目标将由神经形态器件（Roy）、可持续电子纳米材料（Franklin）和神经形态电路和算法（Li）方面的专家承担。目标是：A.开发具有每个像素尽可能高的填充因子的像素内计算电路；B.将像素阵列与基于crossbar的内存计算电路集成，实现用于图像分类分割的CNN；C.采用可回收材料开发整个流程，以减少电子垃圾。对于像素内计算，将设计和优化使用二维材料作为通道和栅极的新型光电突触器件，具有多波长传感能力。像素内计算电路构成CNN的第一层。CNN的后续层将使用记忆突触的横条阵列来实现。材料-器件协同设计将确保突触器件的最佳特性，用于像素内计算和交叉条实现，而器件-系统协同设计将减轻剩余的非理想性，同时通过混合模拟/数字计算方案最大化性能。基于交叉杆电路的器件性能和精度要求，协同设计纳米材料外围电路。将研究并确保所有电路的可回收性，以实现可持续制造。最后，光电突触像素阵列将与横杆阵列和外围电路异质集成，并评估该硬件在图像处理任务（如分类和分割）中的性能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The proposal aims at developing a new technology for artificial intelligence (AI) and machine learning (ML). Using environmentally sustainable materials, a system for processing information – specifically for image processing – will be developed. The current AI models, such as OpenAI’s ChatGPT, require a massive amount of power and resources to run. More companies are rushing to create their ChatGPT clones, which is not sustainable and will only exacerbate climate change. Specialized hardware for vision sensing that is smaller in footprint and more energy-efficient to run AI models on conventional devices like laptops, without relying on cloud servers, is proposed. Artificial vision sensors can transform the future by enabling vision beyond the human eye’s capabilities, leading to impact across the entire landscape of modern life. By detecting changes in motion, it will lead to a new generation of self-driving cars for future “smarter” cities. The proposed vision sensors will enable improved monitoring of processes in the industry, and improved understanding of body movements of athletes, all with low energy consumption and on a small area. Future products enabled by the scientific discoveries and advancements of this work will augment the semiconductor industry. For instance, improved synthesis processes for electronic nanomaterials that are fully recyclable will supplement the industry’s growth, creating new job opportunities. Semiconductor workforce development will be strongly emphasized and enabled by this grant, and students trained through this project will be equipped to join the multitudes of industrial efforts accompanied by the CHIPS for America act and other initiatives. Local college students will be exposed to the developments and trained as a workforce for the semiconductor industry. High school teachers will be exposed to cutting-edge research in semiconductors to convey the enthusiasm of the field to their students.A materials-devices-system co-design for in-sensor computing with pixel arrays heterogeneously integrated with crossbar arrays to form a convolutional neural network (CNN) for image processing, with a special emphasis on sustainable manufacturing, is proposed by the Co-designed Systems for In-sensor Processing with Sustainable Nanomaterials (COSMIC) team. The team will pursue co-design and heterogeneous integration for bio-inspired sensing-to-action to optimize power consumption, performance, and hardware footprint using machine learning architectures. Pursuit of these goals will be undertaken by experts in neuromorphic devices (Roy), nanomaterials for sustainable electronics (Franklin), and neuromorphic circuits and algorithms (Li). The objectives are to: A. Develop in-pixel computing circuits with the highest possible fill factor of each pixel; B. Integrate the pixel arrays with a crossbar-based in-memory computing circuitry to realize a CNN for image classification and segmentation; and C. Develop the entire process with recyclable materials to reduce electronic waste. For in-pixel computing, novel optoelectronic synaptic devices using two-dimensional materials as channel and gate electrode, with multi-wavelength sensing capabilities will be designed and optimized. The in-pixel computing circuit forms the first layer of the CNN. Subsequent layers of the CNN will be realized with crossbar arrays of memristive synapses. Materials-device co-design will ensure the best characteristics of the synaptic devices for in-pixel computing and crossbar implementation, while device-system co-design will mitigate the remaining non-idealities while maximizing performance through a hybrid analog/digital computing scheme. Peripheral circuits with nanomaterials will be co-designed based on device performance and precision requirements of the crossbar circuits. The recyclability of all circuits will be studied and ensured for sustainable manufacturing. Finally, the optoelectronic synapse-based pixel arrays will be heterogeneously integrated with the crossbar arrays and peripheral circuitry, and the performance of this hardware for image processing tasks, such as classification and segmentation, will be evaluated.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteriaThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.