Collaborative Research: CNS Core: Medium: Panoptes: Next Generation Multi-Perspective Video Delivery at Internet Scale

流媒体视频的成功引发了人们对更新形式的多视角视频内容的兴趣，例如由360度摄像机、多角度摄像机阵列或光场摄像机生成的内容。这些摄像头提供的身临其境体验可以提高用户在运动、培训(例如，建筑安全)和虚拟探索(例如，大学漫步、历史遗迹)等领域的满意度。有了这些摄像头的内容，用户不仅可以被动地消费内容，还可以从他们选择的角度沿许多不同的路径交互地遍历内容，不同的用户观察同一内容的不同角度。要在互联网规模上支持这一点是具有挑战性的；客户端播放器必须能够以低延迟切换视角，透视可能需要由视频服务器按需生成，并且基础设施必须支持用于捕获和消费该内容的各种设备。本项目探索架构增强、算法和技术，以在互联网规模上提供多视角视频。它将交付优化与视频编码和人机交互结合在一起。该项目将开发互动性抽象，内容出版商可以通过这些抽象来指定允许用户在视频中的每个点选择的视角范围。视频中指定的互动性将(I)驱动透视编码和新的动态透视生成算法；(Ii)使基础设施供应能够满足内容分发网络(CDN)存储和成本限制；以及(Iii)将引导从CDN服务器或从附近的高速缓存进行自适应透视检索。最后，该项目将探索基于用户研究的预测和指导用户行为的方法，以提高交付质量。通过为互联网规模的多视角视频提供更具响应性和可预测性的用户体验，并通过设计能够应对此类视频增长的网络，该项目将对媒体、IT、教育和网络行业产生影响。它将召集一个由相关从业者组成的工业咨询委员会，并组织研讨会以促进合作。这项研究不仅将广泛涉及研究生，而且还将包括来自代表性不足社区的本科生，将创建新的课程材料，以及一个新的本科设计项目，涉及从一年级到大四的学生，女性和代表性不足的学生有很大的参与。该项目的结果将在以下网站上公布：https://engineering.purdue.edu/~isl/multi-perspective-video.htmland https://nsl.usc.edu/projects/multi-perspective-video/The的结果将在整个项目持续时间内以及项目结束后的三年内提供。该奖项反映了美国国家科学基金会的法定使命，并已通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The success of streaming video has generated interest in newer forms of multi-perspective video content, such as those generated by 360-degree cameras, multi-angle camera arrays, or light-field cameras. The immersive experience provided by these cameras can enhance user satisfaction in domains including sports, training (e.g., construction safety), and virtual exploration (e.g., college walkthroughs, historical sites). With content from these cameras, users do not just passively consume content, but may interactively traverse the content along many different paths from a perspective of their choice, with different users observing different perspectives of the same content. To support this at Internet-scale is challenging; client players must be able to switch perspectives with low latency, the perspectives may need to be generated on demand by video servers, and the infrastructure must support a variety of devices for capture and consumption of this content.This project explores architectural enhancements, algorithms, and techniques to deliver multi-perspective video at Internet-scale. It couples delivery optimization with video coding and human computer interaction. The project will develop interactivity abstractions by which content publishers can specify the range of perspectives users are permitted to choose from at each point in the video. The interactivity specified in a video will (i) drive perspective coding and novel dynamic perspective generation algorithms; (ii) enable infrastructure provisioning to meet Content Delivery Network (CDN) storage and cost constraints; and (iii) will guide adaptive perspective retrieval from CDN servers or from nearby caches. Finally, the project will explore methods to predict and guide user behavior to improve delivery quality based on user studies. By enabling more responsive and predictable user experience for multi-perspective video at Internet scale, and by designing networks that can cope with growth in such video, the project will impact the media, IT, education, and networking industries. It will convene an Industrial Advisory Board comprising relevant practitioners and organize workshops to foster collaborations. The research will not only extensively involve graduate students, but also undergraduate students from underrepresented communities, will create new curriculum material, as well as a new undergraduate design project that involves students ranging from the freshman to senior year with strong participation from women and underrepresented students. The results from the project will be available at: https://engineering.purdue.edu/~isl/multi-perspective-video.htmland https://nsl.usc.edu/projects/multi-perspective-video/The results will be available throughout the duration of the project, and for three years after the project ends.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.