Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
复制标题
DOI：
10.48550/arxiv.2211.03885
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Andrey D. Ignatov;R. Timofte;Shuai Liu;Chaoyu Feng;Furui Bai;Xiaotao Wang;Lei Lei-Lei;Ziyao Yi;Yan Xiang;Zibin Liu;Sha Li;K. Shi;Dehui Kong;Ke Xu;M. Kwon;Yaqi Wu;Jiesi Zheng;Zhihao Fan;Xun Wu;Feng Zhang;Albert No;Minhyeok Cho;Zewen Chen;Xiaze Zhang;Ran Li;Juan Wang;Zhiming Wang;Marcos V. Conde;Ui-Jin Choi;Georgy Perevozchikov;E. Ershov;Zheng Hui;Mengchuan Dong;Xin Lou;Wei Zhou;Cong Pang;Haina Qin;Mingxuan Cai
中科院分区：
文献类型：
--
作者：
Andrey D. Ignatov;R. Timofte;Shuai Liu;Chaoyu Feng;Furui Bai;Xiaotao Wang;Lei Lei-Lei;Ziyao Yi;Yan Xiang;Zibin Liu;Sha Li;K. Shi;Dehui Kong;Ke Xu;M. Kwon;Yaqi Wu;Jiesi Zheng;Zhihao Fan;Xun Wu;Feng Zhang;Albert No;Minhyeok Cho;Zewen Chen;Xiaze Zhang;Ran Li;Juan Wang;Zhiming Wang;Marcos V. Conde;Ui-Jin Choi;Georgy Perevozchikov;E. Ershov;Zheng Hui;Mengchuan Dong;Xin Lou;Wei Zhou;Cong Pang;Haina Qin;Mingxuan Cai
移动的相机的作用在过去几年中急剧增加，导致越来越多的研究自动图像质量增强和RAW照片处理。在这个移动的AI挑战中，目标是开发一个高效的基于端到端AI的图像信号处理（ISP）管道，取代可以使用TensorFlow Lite在现代智能手机GPU上运行的标准移动的ISP。参与者获得了一个大规模的Fujifilm UltraISP数据集，该数据集由普通移动的相机传感器和专业102 MP中画幅Fujifilm GFX 100相机拍摄的数千张配对照片组成。结果模型的运行时在Snapdragon 8 Gen 1 GPU上进行了评估，该GPU为大多数常见的深度学习操作提供了出色的加速结果。所提出的解决方案与所有最新的移动的GPU兼容，能够在不到20-50毫秒的时间内处理全高清照片，同时实现高保真效果。本文提供了在这一挑战中开发的所有模型的详细描述。
The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.