US Ignite: Focus Area 1: Predictable Wireless Networking and Collaborative 3D Reconstruction for Real-Time Augmented Vision

消除了人类视觉和机器视觉的视线限制，基于可预测的无线联网和3D重建的开发的网络系统基础将实现透明视觉，这将改变人类和工程系统与环境交互的方式，从而对道路交通、公共安全和灾难应对等领域产生深远影响。该项目为配备传感器和增强现实显示器的车辆开发网络系统基础，以指示附近被障碍物隐藏的其他车辆的存在。该项目与韦恩州立大学(WSU)警方和福特研究公司合作，利用WSU生活实验室和OpenXC互联汽车开源平台，将采用集成的方法来研究、部署和传播无线网络系统，以实现透视愿景。该项目提出了一个跨层框架，用于解决物理域的不确定性以及无线网络和3D重建之间的相互依赖关系，并为可预测的无线网络和实时无线网络三维重建开发了新的算法。利用开发的网络系统，该项目将开发一个用于人类驾驶的透视视觉应用程序。无线联网透视视觉系统将部署在西德克萨斯州立大学的警察巡逻车上，项目团队将与底特律和密歇根州警方以及开源社区进行接触，以广泛采用和部署透视视觉系统。该项目以消除人类视觉和机器视觉的视线限制为大胆目标，在整体跨层框架中解决无线联网和3D重建挑战。通过将研究调查与系统开发和部署相结合，本项目将做出以下重大贡献：1)有效地利用交通流的多尺度物理结构，车载无线网络中的多尺度资源管理方法不仅确保可预测的车载无线网络，而且将车载网络的基本挑战转变为类似于大多数固定网络的挑战，从而使探索可预测的无线网络的基本、普遍适用的原则和机制成为可能；2)联合调度、信道分配、功率控制和速率控制的多尺度方法能够在快速变化的网络和环境条件(如无线信道衰减、内部和外部干扰、数据流量动态和车辆移动性)下对每个分组的传输可靠性进行可预测的控制；3)实时调度算法使得能够可控地探索实时容量区域以进行系统级优化；4)协作三维重建模型以分而治之的方式集成视觉传感器，增强了网络视觉的能力和对物理不确定性的鲁棒性；5)协同3D重建和无线网络的联合设计允许自适应的通信容量分配以优化3D重建质量；6)注意力感知透视视觉应用独特地将计算机视觉和计算机图形学研究相结合，并提出了显示增强的3D视觉的实用解决方案，从而开创了视觉增强的新研究领域。

Eliminating the line-of-sight constraint of human vision and machine vision, the developed network systems foundations of predictable wireless networked and 3D reconstruction will enable 'see-through vision' which will transform the ways humans and engineered systems interact with environments and thus have far-reaching impact on domains such as road transportation, public safety, and disaster response. This project develops the network systems foundation for a vehicle equipped with sensors and an augmented reality display to indicate the presence of other nearby vehicles hidden by obstacles. In collaboration with Wayne State University (WSU) police and Ford Research and leveraging the WSU living lab and the OpenXC open-source platform for connected vehicles, the project will take an integrated approach to the research, deployment, and dissemination of the wireless network systems for see-through vision. This project proposes a cross-layer framework for addressing physical-domain uncertainties and the interdependencies between wireless networking and 3D reconstruction, and it develops novel algorithms for predictable wireless networking and real-time wireless networked 3D reconstruction. Using the developed network system, this project will develop a see-through vision application for human-driving. The wireless networked see-through vision system will be deployed in the WSU police patrol vehicles, and the project team will outreach to the Detroit and State of Michigan police as well as open-source communities for broad adoption and deployment of the see-through vision system.With the bold objective of eliminating the line-of-sight constraint of human vision and machine vision, this project addresses wireless networking and 3D reconstruction challenges in a holistic cross-layer framework. By integrating research investigation with systems development and deployment, this project will make the following significant contributions: 1) Effectively leveraging multi-scale physical structures of traffic flows, the multi-scale approach to resource management in vehicular wireless networks not only ensures predictable vehicular wireless networking, it also transforms fundamental challenges of vehicular networks to ones similar to those of mostly-immobile networks, thus enabling the exploration of fundamental, generically-applicable principles and mechanisms for predictable wireless networking; 2) the multi-scale approach to joint scheduling, channel assignment, power control, and rate control enables predictable control of per-packet transmission reliability in the presence of fast-varying network and environmental conditions such as wireless channel attenuation, internal and external interference, data traffic dynamics, and vehicle mobility; 3) the real-time scheduling algorithm enables controllable exploration of real-time capacity regions for system-level optimization; 4) the collaborative 3D reconstruction model integrates visual sensors in a divide-and-conquer fashion, and it enhances the capability of networked vision as well as its robustness to physical uncertainties; 5) the co-design of collaborative 3D reconstruction and wireless networking permits adaptive communication capacity allocation to optimize the quality of 3D reconstruction; 6) the attention-aware see-through vision application creates a new research field of vision augmentation by uniquely integrating computer vision and computer graphics research and by proposing a practical solution for displaying augmented 3D vision.