Acquisition and Coding of Light-Ray Data for 3D Spatial Communication

在这个项目下，我们研究了一些光线数据的获取和编码方法，作为三维空间通信的一组基本元素。1.面向三维空间通信的光线数据描述方法三维空间的视觉数据可以表示为一个称为光场或光线空间的四维函数，其中填充三维空间的光线被视为基本元素。我们将频域分析引入光线空间，描述了多视点图像的采集、光线数据的分析和自由视点图像的合成过程。特别是，我们提出了聚焦测量法来识别合成图像上清晰的部分，这是将信号处理理论与计算机视觉技术相结合的重要贡献。2.高效获取三维空间数据的光线数据获取方法获取光线数据相当于对光线空间进行采样。在这个项目中，我们CLA…进一步明确了采样条件与采样数据合成图像质量之间的关系。为了追求理想的采样条件，我们与NHK科学与技术研究实验室合作，研究了一种使用数千个小透镜阵列的多视角采集方法。3.光线数据的数据压缩方法，以实现三维空间数据的高效传输。为了压缩光线数据，需要考虑对多种应用的适应性以及对数据中冗余的开发。在本项目中，我们重点研究了用于合成自由视点图像的参考局部性，并利用该局部性对一种高效的编码方案进行了基础研究。4.基于光线数据的3D空间通信原型实时处理和低延迟是3D空间通信的重要要求。提出并实现了一种无需离线处理的实时合成自由视点图像的算法，成功地实现了动态场景的实时处理。较少

Under this project, we investigated some methods for acquisition and coding of light-ray data as a set of fundamental elements for 3-D spatial communication.1.Description method of light-ray data towards 3D spatial communicationVisual data of a 3-D space can be represented as a 4-D function called a light field or a ray space, in which the light-rays filling the 3-D space are regarded as a fundamental element. We introduced the frequency domain analysis to the ray space in order to describe the processes of acquisition of multi-view images, analysis on the light-ray data, and synthesis of free-viewpoint images. Especially, we proposed the "focus measurement method" to identify clear parts on synthesized images, which is an important contribution that relates a signal processing theory with computer vision technologies.2.Acquisition method of light-ray data for efficient capturing of 3D spatial dataCapturing light-ray data is equivalent to sampling the ray space. In this project, we cla … More rified the relation between the sampling condition and the quality of the images synthesized from the sampled data. To pursue the ideal sampling condition, we investigated a multi-view acquisition method using an array of thousands of lenslets in cooperation with NHK Science and Technical Research Laboratories.3.Data compression method for light-ray data for efficient transmission of 3D spatial dataFor compressing the light-ray data, we should consider the adaptability to several applications as well as the exploitation of the redundancy in the data. In this project, we focused on the "locality of reference" for synthesizing free-viewpoint images, and conducted a basic study on an efficient coding scheme using this locality.4.Prototype of 3D spatial communication based on light-ray dataReal-time processing and low-latency are the important requirements for 3D spatial communication. We proposed and implemented a real-time algorithm for synthesizing free-viewpoint images without any off-line processing, and succeeded in real time processing of dynamic scenes. Less