CAREER: Scarlet: Learned Protocols and Functional Architectures for Low-Latency Internet Video

该项目将创造新的方法来设计在互联网上真实的传输视频和音频的系统，改进商业最先进的技术（例如，Zoom、Google Meet、Skype或FaceTime）。该项目的假设是，学术计算机科学中的新技术（如纯功能架构，机器学习和低延迟实时分布式系统设计）已经准备好成功应用于这一领域。最终的目标是创造新的系统，让人们感觉他们在同一个地方，能够表演音乐或戏剧，或者在互联网上进行强大的对话。该项目计划在三个领域进行科学调查：协议，编解码器和压缩。对于协议，其核心思想是应用模型预测控制和原位学习来研究单个网络路径，并以细粒度预测其逐包动态，以控制近期行为。对于编解码器，主要假设是将机器学习紧密结合在编码器的内部速率控制算法中可以减少端到端延迟，并产生对传输协议有用的代理目标函数。如果这些努力取得成功，这就为压缩提供了机会：低延迟可以用来代替带宽，实时凝视信息和视凝视而定的压缩可以带来多倍的压缩增益和360 x180的“视网膜质量”（虚拟现实）视频流在现实的路径能力。该项目将包括在一系列现场在线戏剧和音乐表演，通过互联网连接的演员和观众。表演将作为一个学习的机会和研究项目的强制功能，一个机会，为表演者谁不能旅行的物理配置，并在日常设置的计算机科学研究的好处公开演示。该项目将包括为斯坦福大学的本科生开设有关视频和音频传输技术发展的新课程。该项目将在https://github.com/stanford-stagecast上维护一个源代码库。代码将在编写时公开发布;数据和结果将在学术文献中发表，也将公开提供（从存储库链接）。这个奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为是值得支持的。

This project will create new approaches to the design of systems that transmit video and audio over the Internet in real time, improving on the commercial state-of-the-art (e.g., Zoom, Google Meet, Skype, or FaceTime). The hypothesis underlying this project is that new techniques in academic computer science (such as purely functional architectures, machine learning, and low-latency real-time distributed systems design) are ready to be applied successfully to this area. The ultimate goal will be to create new systems that allow people to feel like they are in the same place, able to perform music or theater or have a robust conversation while separated over the Internet.The project plans to pursue scientific inquiries in three areas: protocols, codecs, and compression. For protocols, the core idea is to apply Model-Predictive Control and in-situ learning to study an individual network path and predict its packet-by-packet dynamics at a fine grain, in order to control near-term behavior. For codecs, the main hypothesis is that incorporating machine learning tightly in an encoder's inner rate-control algorithm can reduce end-to-end latency and produce a proxy objective function that is useful to the transport protocol. If these efforts are successful, this opens up an opportunity for compression: the idea that low latency can be used to substitute for bandwidth, and that real-time gaze information and gaze-contingent compression can lead to manyfold compression gains and "retina-quality" 360x180 (virtual reality) video streaming over realistic path capacities.The project will include in a series of live online theatrical and musical performances, using far-flung actors and audience members connected over the Internet. The performances will serve as a learning opportunity and forcing function for the research project, an opportunity for performers who can't travel to be physically collocated, and a public demonstration of the benefits of computer-science research in an everyday setting. The project will include new classes taught for undergraduates at Stanford University on the development of video and audio transmission technology.The project will maintain a source-code repository at https://github.com/stanford-stagecast. Code will be released publicly as it is written; data and results will be published in the academic literature and will be available publicly (linked from the repository) as well. The project website will be maintained for at least three years.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.