ITR - Remote Reality: 4-D Audio-Visual Reconstruction and Compression from Multiple Sensors

ITR - 远程现实：来自多个传感器的 4D 视听重建和压缩

• 批准号: 0312432
• 负责人: Douglas Jones
• 金额: $ 32.67万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2003
• 资助国家: 美国
• 起止时间: 2003-07-15 至 2007-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0312432&HistoricalAwards=false
• 关键词: ITR; Remote; Reality; Audio; Visual

Abstract0312432Douglas JonesU of Illinois, Urbana-ChampaignCurrent audiovisual recording, communication, and playback provides a single, two-dimensional perspectiveof the world as it varies through time. Humans live in a \four-dimensional" world, in which they move about(in 3-D space over time) at their own volition to experience the world from any perspective. Advances insensor, computer, and networking technologies now make possible new systems that employ multiple cam-eras and microphones together with sophisticated processing algorithms to deliver unprecedented immersiverecording and viewing capabilities. Sufficient sensing, networking, and computing power to practically ad-dress this vision already exists; the critical gap in achieving it is the lack of the necessary signal processingtheory and algorithms.This research will develop new signal processing techniques for reconstruction of the audio and visualrecording at an arbitrary location in space and time from multiple acoustic and video sensors, by extendingrecent research in adaptive beamforming, multisensor signal processing of non-stationary signals, and fun-damental new advances in multi-dimensional signal representation. Practical four-dimensional audiovisualrecording, transmission, and playback, or \remote reality", will be demonstrated with low-cost, conven-tional sensors attached to networked computers, thus confirming the practicality of the proposed methodsand applications.

[摘要]伊利诺斯州厄巴纳-香槟分校的道格拉斯·琼斯目前的视听记录、交流和播放提供了一个单一的、二维的视角，因为它随着时间的推移而变化。人类生活在一个“四维”的世界里，在这个世界里，他们按照自己的意愿（随着时间的推移，在三维空间中）四处移动，从任何角度体验世界。传感器、计算机和网络技术的进步使新系统成为可能，该系统采用多个摄像头和麦克风以及复杂的处理算法，提供前所未有的沉浸式记录和观看功能。已经存在足够的传感、网络和计算能力来实际实现这一愿景；实现这一目标的关键差距是缺乏必要的信号处理理论和算法。本研究将通过扩展自适应波束形成、非平稳信号的多传感器信号处理和多维信号表示的基本新进展，开发新的信号处理技术，用于从多个声学和视频传感器重建在空间和时间任意位置的音频和视频记录。实用的四维视听记录、传输和回放，或“远程现实”，将通过连接到联网计算机上的低成本传统传感器进行演示，从而确认所提出方法和应用的实用性。