IMR: MM-1C: Enabling Continual Passive Estimation of Performance of Internet Transfers: Online Measurement and Classification Methods

该项目的重点是开发可以监控网络流量并可靠地估计互联网用户体验的性能的技术，同时只查看网络数据包的报头，而不查看任何侵犯隐私的信息。它们的主要新奇还在于推断什么类型的网络连接、什么类型的应用程序以及用户正在使用什么类型的设备来访问互联网，并且能够足够快地进行所有分析以跟上互联网承载的流量的高速。该项目提出的创新包括：（i）设计深度学习框架，根据接入网络、客户端平台和应用程序使用情况对最终用户细分进行分类，这将增强对互联网性能的理解。(ii)在线采样、散列和草图技术的设计，这将使其他类型的被动分析也能够以在线和轻量级的方式进行。目前，大多数被动分析研究都被降级为存储和处理大型跟踪集，这限制了它们的部署。(iii)被动估计网络带宽是否限制了互联网传输，这在像互联网这样多样化和不断发展的环境中从未尝试过。这个合作项目汇集了来自计算机科学和统计学领域的研究人员，预计将改变几个领域：（i）拟议的用户分类工作还将有助于了解互联网的其他特性（不仅仅是性能）在不同用户分类之间的差异。这将有助于确保所有公民都能公平和普遍地使用互联网。 (ii)建议的监测技术将是一个重要的来源，洞察网络系统管理，可以帮助缓解系统瓶颈，提高性能。 (iii)在大数据的实验、测量和科学分析方面的经验对于参与大型数据集信息挖掘的联邦、商业和学术机构来说是非常宝贵的。在模拟和分析大量的交通数据的建议的努力将是一个很好的来源，本科生和研究生在这些方面的培训。(iv)建议的让少数民族和本科生参与研究的努力将有助于扩大数据科学（计算机科学/统计学）工作队伍的多样性和能力。(v)拟议的外展工作（面向初中和高中生）将有助于增加社区对科学和技术的参与，该项目的网址是：https://sites.google.com/cs.unc.edu/real-time-passive-traffic-anal/home。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project focuses on developing techniques that can monitor network traffic and reliably estimate the performance experienced by Internet users, while only looking at headers of network packets and not looking at any privacy-violating information. They main novelty is in also inferring what type of network connection, what type of application, and what type of device the user is using for accessing the Internet, and in being able to do all of the analysis fast enough to keep up with high speed with which traffic is carried by the Internet. The innovations proposed in this project include: (i) The design of deep learning frameworks for classifying end-user segments, based on access networks, client platforms, and application usage, which will enhance understanding of Internet performance. (ii) The design of online sampling, hashing, and sketching techniques, which will enable other types of passive analysis to also be conducted in an online and light-weight manner. Currently, most passive analysis studies are relegated to storing and working with large sets of traces, which limits the their deployment. (iii) Passive estimation of whether network bandwidth constrains an Internet transfer, which has never been attempted before in a setting as diverse and evolving as the Internet.This collaborative project brings together investigators from the fields of Computer Science and Statistics, and is expected to transform several domains: (i) The proposed efforts in user segmentation will also aid in understanding how other properties of the Internet (not just performance) differ across different user segments. This will aid in ensuring equitable and ubiquitous Internet access for all citizens. (ii) The proposed monitoring techniques will be an important source of insights on network systems management that can help alleviate system bottlenecks and improve performance. (iii) Experience in experimentation, measurements, and scientific analysis of big data is invaluable to federal, commercial, and academic institutions that are involved in mining for information in large data-sets. The proposed efforts in emulation and analysis of massive amounts of traffic data will be an excellent source of undergraduate and graduate students trained in these aspects. (iv) The proposed efforts in involving minorities and undergraduates in research will help broaden the diversity and capabilities of the Data Science (Computer Science/Statistics) work force. (v) The proposed outreach efforts (to middle- and high-schoolers) will help increase community engagement with science and technology.The website for this project can be found at: https://sites.google.com/cs.unc.edu/real-time-passive-traffic-anal/home. This website will be updated with new developments and resources, until the end of the project activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.