IMR: MM-1C: Enabling Continual Passive Estimation of Performance of Internet Transfers: Online Measurement and Classification Methods

该项目专注于开发能够监控网络流量并可靠地估计互联网用户体验的性能的技术，同时只查看网络分组的报头，而不查看任何侵犯隐私的信息。它们的主要新颖性还在于推断用户正在使用哪种类型的网络连接、哪种类型的应用程序和哪种类型的设备访问互联网，并且能够足够快地完成所有分析，以跟上互联网承载流量的高速。该项目提出的创新包括：(I)设计深度学习框架，根据接入网络、客户端平台和应用程序使用情况对终端用户进行分类，这将加强对互联网性能的了解。(Ii)设计在线采样、散列和草图技术，这将使其他类型的被动分析也能够以在线和轻量级的方式进行。目前，大多数被动分析研究都归因于存储和处理大量痕迹，这限制了它们的部署。(Iii)对网络带宽是否限制互联网传输的被动估计，这是以前从未尝试过的，在像互联网这样多样化和不断发展的环境中。这一合作项目汇集了来自计算机科学和统计领域的研究人员，预计将改变几个领域：(I)拟议的用户细分努力也将有助于理解互联网的其他属性(不仅仅是性能)在不同用户群之间的差异。这将有助于确保所有公民都能公平和无处不在地接入互联网。(2)拟议的监测技术将是深入了解网络系统管理的重要来源，有助于缓解系统瓶颈和提高性能。(Iii)在大数据的实验、测量和科学分析方面的经验对于参与大数据信息挖掘的联邦、商业和学术机构来说是非常宝贵的。拟议的大量交通数据仿真和分析工作将成为在这些方面进行培训的本科生和研究生的极好来源。(Iv)让少数族裔和本科生参与研究的拟议努力将有助于扩大数据科学(计算机科学/统计)工作队伍的多样性和能力。(V)(针对初中生和高中生)拟议的外展工作将有助于提高社区对科学和技术的参与度。该项目的网址为：https://sites.google.com/cs.unc.edu/real-time-passive-traffic-anal/home.本网站将更新新的发展和资源，直到项目活动结束。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project focuses on developing techniques that can monitor network traffic and reliably estimate the performance experienced by Internet users, while only looking at headers of network packets and not looking at any privacy-violating information. They main novelty is in also inferring what type of network connection, what type of application, and what type of device the user is using for accessing the Internet, and in being able to do all of the analysis fast enough to keep up with high speed with which traffic is carried by the Internet. The innovations proposed in this project include: (i) The design of deep learning frameworks for classifying end-user segments, based on access networks, client platforms, and application usage, which will enhance understanding of Internet performance. (ii) The design of online sampling, hashing, and sketching techniques, which will enable other types of passive analysis to also be conducted in an online and light-weight manner. Currently, most passive analysis studies are relegated to storing and working with large sets of traces, which limits the their deployment. (iii) Passive estimation of whether network bandwidth constrains an Internet transfer, which has never been attempted before in a setting as diverse and evolving as the Internet.This collaborative project brings together investigators from the fields of Computer Science and Statistics, and is expected to transform several domains: (i) The proposed efforts in user segmentation will also aid in understanding how other properties of the Internet (not just performance) differ across different user segments. This will aid in ensuring equitable and ubiquitous Internet access for all citizens. (ii) The proposed monitoring techniques will be an important source of insights on network systems management that can help alleviate system bottlenecks and improve performance. (iii) Experience in experimentation, measurements, and scientific analysis of big data is invaluable to federal, commercial, and academic institutions that are involved in mining for information in large data-sets. The proposed efforts in emulation and analysis of massive amounts of traffic data will be an excellent source of undergraduate and graduate students trained in these aspects. (iv) The proposed efforts in involving minorities and undergraduates in research will help broaden the diversity and capabilities of the Data Science (Computer Science/Statistics) work force. (v) The proposed outreach efforts (to middle- and high-schoolers) will help increase community engagement with science and technology.The website for this project can be found at: https://sites.google.com/cs.unc.edu/real-time-passive-traffic-anal/home. This website will be updated with new developments and resources, until the end of the project activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.