ITR - Remote Reality: 4-D Audio-Visual Reconstruction and Compression from Multiple Sensors

点击翻译按钮获取中文摘要

Abstract0312432Douglas JonesU of Illinois, Urbana-ChampaignCurrent audiovisual recording, communication, and playback provides a single, two-dimensional perspectiveof the world as it varies through time. Humans live in a \four-dimensional" world, in which they move about(in 3-D space over time) at their own volition to experience the world from any perspective. Advances insensor, computer, and networking technologies now make possible new systems that employ multiple cam-eras and microphones together with sophisticated processing algorithms to deliver unprecedented immersiverecording and viewing capabilities. Sufficient sensing, networking, and computing power to practically ad-dress this vision already exists; the critical gap in achieving it is the lack of the necessary signal processingtheory and algorithms.This research will develop new signal processing techniques for reconstruction of the audio and visualrecording at an arbitrary location in space and time from multiple acoustic and video sensors, by extendingrecent research in adaptive beamforming, multisensor signal processing of non-stationary signals, and fun-damental new advances in multi-dimensional signal representation. Practical four-dimensional audiovisualrecording, transmission, and playback, or \remote reality", will be demonstrated with low-cost, conven-tional sensors attached to networked computers, thus confirming the practicality of the proposed methodsand applications.