Joint neural phase retrieval and compression for energy- and computation-efficient holography on the edge
Joint neural phase retrieval and compression for energy- and computation-efficient holography on the edge
复制标题
DOI：
10.1145/3528223.3530070
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Yujie Wang;Praneeth Chakravarthula;Qingyan Sun;Baoquan Chen
中科院分区：
文献类型：
--
作者：
Yujie Wang;Praneeth Chakravarthula;Qingyan Sun;Baoquan Chen
最近的深度学习方法已经显示出实现高保真全息显示的巨大潜力。然而，由于有限的板载计算能力和电池寿命，轻量级可穿戴显示设备无法承受全息图生成的计算需求和能量消耗。另一方面，如果计算完全在云服务器上远程进行，则传输无损全息数据不仅具有挑战性，而且会导致高延迟和高存储。在这项工作中，通过分配计算和优化传输，我们提出了第一个联合生成和压缩高质量纯相位全息图的框架。具体来说，我们的框架不对称地将全息图生成过程分为高计算远程编码（在服务器上）和低计算解码（在边缘上）阶段。我们的编码实现了轻量级的潜在空间数据，从而更快，更有效地传输到边缘设备。通过我们的框架，我们观察到与现有的全息图生成方法相比，边缘设备上的计算减少了76%，因此能源成本减少了83%。我们的框架是强大的传输和解码错误，并接近高图像保真度低至2位每像素，并进一步降低平均比特率和解码时间的全息视频。
Recent deep learning approaches have shown remarkable promise to enable high fidelity holographic displays. However, lightweight wearable display devices cannot afford the computation demand and energy consumption for hologram generation due to the limited onboard compute capability and battery life. On the other hand, if the computation is conducted entirely remotely on a cloud server, transmitting lossless hologram data is not only challenging but also result in prohibitively high latency and storage. In this work, by distributing the computation and optimizing the transmission, we propose the first framework that jointly generates and compresses high-quality phase-only holograms. Specifically, our framework asymmetrically separates the hologram generation process into high-compute remote encoding (on the server), and low-compute decoding (on the edge) stages. Our encoding enables light weight latent space data, thus faster and efficient transmission to the edge device. With our framework, we observed a reduction of 76% computation and consequently 83% in energy cost on edge devices, compared to the existing hologram generation methods. Our framework is robust to transmission and decoding errors, and approach high image fidelity for as low as 2 bits-per-pixel, and further reduced average bit-rates and decoding time for holographic videos.