IMR: MT: Tools for Safe, Easy, and Reliable Active Global Internet Measurement

点击翻译按钮获取中文摘要

It is difficult to predict Internet behavior due the complexity and distributed nature of the Internet. For example, it is hard to know whether proposed Internet protocols will work when deployed. As a result, researchers turn to large-scale, measurement of real-world systems to understand how the Internet is structured, how protocols operate, and where there are security and performance failures. This proposal improves a core set of measurement tools that researchers use for performing large-scale measurement of Internet hosts, websites, names, cloud assets to prevent common measurement errors, support recently introduced protocols, and account for real-world deployment complexities.This proposal will specifically make improvements to the ZMap suite of active measurement tools, including to (1) automatically detect error conditions and prevent erroneous data, (2) adapt tools to accommodate recently introduced network protocols like TLS 1.3, and (3) enable tools to find and accommodate services that appear in unpredictable locations. The challenge in these tasks lays in the fact that: (1) because there is no ground truth against which to compare results, tools need to intelligently determine whether results have become unstable and to alter their behavior while maintaining data consistency, (2) must operate with exceptional performance to accommodate the size of Internet-wide studies, and (3) must produce and structure data that researchers can use when research tasks are not known ahead of time.The impact of this project lies in the research that it will enable. ZMap has been used in many research publications over the past few years, but research has uncovered several shortcomings and error conditions that can produce inconsistent data. The Internet has also evolved over this time period, which requires software changes to accommodate. The improvements will help make ZMap accessible to additional researchers, ensure that researchers with less Internet measurement experience can reliably collect valid results, enable the study of recently introduced protocols, and enable researchers to use the suite of tools as part of broader measurement platforms.The ZMap tools are documented and available for download at https://zmap.io. Any auxiliary datasets produced will be published at https://scans.io. ZMap is planned to be maintained for the foreseeable future.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.