US Ignite: Focus Area 1: Predictable Wireless Networking and Collaborative 3D Reconstruction for Real-Time Augmented Vision

消除了人类视觉和机器视觉的视线限制，可预测的无线网络和3D重建的网络系统基础将实现“透视视觉”，这将改变人类和工程系统与环境交互的方式，从而对道路运输，公共安全和灾难响应等领域产生深远的影响。该项目为配备传感器和增强现实显示器的车辆开发网络系统基础，以指示附近被障碍物隐藏的其他车辆的存在。 该项目与韦恩州立大学（WSU）警察局和福特研究院合作，利用WSU生活实验室和OpenXC互联车辆开源平台，将采取综合方法研究、部署和传播用于透视视觉的无线网络系统。该项目提出了一个跨层框架，用于解决物理域的不确定性和无线网络和3D重建之间的相互依赖性，并开发了可预测的无线网络和实时无线网络3D重建的新算法。利用所开发的网路系统，本计画将开发一个用于人类驾驶的透视视觉应用。该无线网络透视视觉系统将部署在WSU警察巡逻车上，项目团队将与底特律和密歇根州警方以及开源社区进行外联，以广泛采用和部署透视视觉系统。凭借消除人类视觉和机器视觉的视线限制的大胆目标，该项目在一个整体的跨层框架中解决了无线网络和3D重建的挑战。通过将研究调查与系统开发和部署相结合，该项目将做出以下重大贡献：1）有效地利用业务流的多尺度物理结构，车辆无线网络中的资源管理的多尺度方法不仅确保了可预测的车辆无线联网，还将车辆网络的基本挑战转变为类似于大多数固定网络的挑战，从而使得能够探索用于可预测的无线联网的基本的、普遍适用的原理和机制; 2）联合调度、信道分配、功率控制和速率控制的多尺度方法使得能够在存在快速变化的网络和环境条件（例如无线信道衰减、内部和外部干扰、数据业务动态性）的情况下对每分组传输可靠性进行可预测的控制，车辆机动性; 3）实时调度算法实现了对实时容量区域的可控探索，实现了系统级优化：4）协同三维重建模型以分而治之的方式集成了多个视觉传感器，增强了网络视觉的能力和对物理不确定性的鲁棒性; 5）协同3D重建和无线联网的协同设计允许自适应通信容量分配以优化3D重建的质量; 6）注意力感知透视视觉应用通过独特地集成计算机视觉和计算机图形学研究并通过提出用于显示增强的3D视觉的实用解决方案来创建视觉增强的新研究领域。

Eliminating the line-of-sight constraint of human vision and machine vision, the developed network systems foundations of predictable wireless networked and 3D reconstruction will enable 'see-through vision' which will transform the ways humans and engineered systems interact with environments and thus have far-reaching impact on domains such as road transportation, public safety, and disaster response. This project develops the network systems foundation for a vehicle equipped with sensors and an augmented reality display to indicate the presence of other nearby vehicles hidden by obstacles. In collaboration with Wayne State University (WSU) police and Ford Research and leveraging the WSU living lab and the OpenXC open-source platform for connected vehicles, the project will take an integrated approach to the research, deployment, and dissemination of the wireless network systems for see-through vision. This project proposes a cross-layer framework for addressing physical-domain uncertainties and the interdependencies between wireless networking and 3D reconstruction, and it develops novel algorithms for predictable wireless networking and real-time wireless networked 3D reconstruction. Using the developed network system, this project will develop a see-through vision application for human-driving. The wireless networked see-through vision system will be deployed in the WSU police patrol vehicles, and the project team will outreach to the Detroit and State of Michigan police as well as open-source communities for broad adoption and deployment of the see-through vision system.With the bold objective of eliminating the line-of-sight constraint of human vision and machine vision, this project addresses wireless networking and 3D reconstruction challenges in a holistic cross-layer framework. By integrating research investigation with systems development and deployment, this project will make the following significant contributions: 1) Effectively leveraging multi-scale physical structures of traffic flows, the multi-scale approach to resource management in vehicular wireless networks not only ensures predictable vehicular wireless networking, it also transforms fundamental challenges of vehicular networks to ones similar to those of mostly-immobile networks, thus enabling the exploration of fundamental, generically-applicable principles and mechanisms for predictable wireless networking; 2) the multi-scale approach to joint scheduling, channel assignment, power control, and rate control enables predictable control of per-packet transmission reliability in the presence of fast-varying network and environmental conditions such as wireless channel attenuation, internal and external interference, data traffic dynamics, and vehicle mobility; 3) the real-time scheduling algorithm enables controllable exploration of real-time capacity regions for system-level optimization; 4) the collaborative 3D reconstruction model integrates visual sensors in a divide-and-conquer fashion, and it enhances the capability of networked vision as well as its robustness to physical uncertainties; 5) the co-design of collaborative 3D reconstruction and wireless networking permits adaptive communication capacity allocation to optimize the quality of 3D reconstruction; 6) the attention-aware see-through vision application creates a new research field of vision augmentation by uniquely integrating computer vision and computer graphics research and by proposing a practical solution for displaying augmented 3D vision.