IMR: MM-1C: Enabling Continual Passive Estimation of Performance of Internet Transfers: Online Measurement and Classification Methods

该项目侧重于开发能够监控网络流量并可靠地估计Internet用户所经历的性能的技术，同时只查看网络数据包的标头，而不查看任何侵犯隐私的信息。它们的主要新颖之处还在于推断用户使用何种类型的网络连接、何种类型的应用程序和何种类型的设备来访问互联网，并且能够快速完成所有分析，以跟上互联网承载的高速流量。本项目提出的创新包括：(i)基于接入网络、客户端平台和应用程序使用情况，设计用于对最终用户细分进行分类的深度学习框架，这将增强对互联网性能的理解。（ii）在线采样、散列和草图技术的设计，这将使其他类型的被动分析也能以在线和轻量级的方式进行。目前，大多数被动分析研究都被降级为存储和处理大量的跟踪，这限制了它们的部署。（iii）被动估计网络带宽是否限制了互联网传输，这在互联网这样多样化和不断发展的环境中从未尝试过。这个合作项目汇集了来自计算机科学和统计学领域的研究人员，并有望改变几个领域：(i)在用户细分方面提出的努力也将有助于理解互联网的其他属性（不仅仅是性能）在不同用户细分之间的差异。这将有助于确保所有公民公平和无处不在的互联网接入。拟议的监测技术将是了解网络系统管理的重要来源，有助于减轻系统瓶颈和改善性能。（iii）大数据的实验、测量和科学分析经验对于参与挖掘大型数据集信息的联邦、商业和学术机构来说是非常宝贵的。在模拟和分析大量交通数据方面所提出的努力将是在这些方面受过训练的本科生和研究生的极好来源。（iv）建议让少数族裔和本科生参与研究的努力将有助于扩大数据科学（计算机科学/统计）工作人员的多样性和能力。拟议的外联努力（面向初中生和高中生）将有助于增加社区对科学和技术的参与。这个项目的网站可以在https://sites.google.com/cs.unc.edu/real-time-passive-traffic-anal/home上找到。本网站将不断更新新的发展和资源，直至项目活动结束。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project focuses on developing techniques that can monitor network traffic and reliably estimate the performance experienced by Internet users, while only looking at headers of network packets and not looking at any privacy-violating information. They main novelty is in also inferring what type of network connection, what type of application, and what type of device the user is using for accessing the Internet, and in being able to do all of the analysis fast enough to keep up with high speed with which traffic is carried by the Internet. The innovations proposed in this project include: (i) The design of deep learning frameworks for classifying end-user segments, based on access networks, client platforms, and application usage, which will enhance understanding of Internet performance. (ii) The design of online sampling, hashing, and sketching techniques, which will enable other types of passive analysis to also be conducted in an online and light-weight manner. Currently, most passive analysis studies are relegated to storing and working with large sets of traces, which limits the their deployment. (iii) Passive estimation of whether network bandwidth constrains an Internet transfer, which has never been attempted before in a setting as diverse and evolving as the Internet.This collaborative project brings together investigators from the fields of Computer Science and Statistics, and is expected to transform several domains: (i) The proposed efforts in user segmentation will also aid in understanding how other properties of the Internet (not just performance) differ across different user segments. This will aid in ensuring equitable and ubiquitous Internet access for all citizens. (ii) The proposed monitoring techniques will be an important source of insights on network systems management that can help alleviate system bottlenecks and improve performance. (iii) Experience in experimentation, measurements, and scientific analysis of big data is invaluable to federal, commercial, and academic institutions that are involved in mining for information in large data-sets. The proposed efforts in emulation and analysis of massive amounts of traffic data will be an excellent source of undergraduate and graduate students trained in these aspects. (iv) The proposed efforts in involving minorities and undergraduates in research will help broaden the diversity and capabilities of the Data Science (Computer Science/Statistics) work force. (v) The proposed outreach efforts (to middle- and high-schoolers) will help increase community engagement with science and technology.The website for this project can be found at: https://sites.google.com/cs.unc.edu/real-time-passive-traffic-anal/home. This website will be updated with new developments and resources, until the end of the project activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.