IMR: MT: Tools for Safe, Easy, and Reliable Active Global Internet Measurement

由于互联网的复杂性和分布性，很难预测互联网行为。例如，很难知道拟议的互联网协议在部署时是否会起作用。因此，研究人员转向对真实世界系统的大规模测量，以了解互联网是如何构建的，协议是如何运作的，以及哪里存在安全和性能故障。该建议改进了研究人员用于对互联网主机、网站、名称、云资产进行大规模测量的一组核心测量工具，以防止常见的测量错误，支持最近引入的协议，并考虑到现实世界的部署复杂性。该建议将具体对ZMap主动测量工具套件进行改进，包括(1)自动检测错误条件和防止错误数据，(2)调整工具以适应最近引入的网络协议，如TLS 1.3，以及(3)使工具能够发现和适应出现在不可预测位置的服务。这些任务的挑战在于：(1)由于没有可以比较结果的基本事实，工具需要智能地确定结果是否变得不稳定，并在保持数据一致性的同时改变它们的行为；(2)必须以卓越的性能运行，以适应互联网范围内研究的规模；以及(3)必须产生和组织研究人员在研究任务未知的情况下可以使用的数据。这个项目的影响在于它将实现的研究。在过去的几年里，ZMap在许多研究出版物中被使用，但研究发现了几个可能产生不一致数据的缺陷和错误情况。互联网也在这段时间内发展，这需要软件的变化来适应。这些改进将有助于使更多的研究人员能够访问Zmap，确保缺乏互联网测量经验的研究人员能够可靠地收集有效的结果，使研究人员能够研究最近引入的协议，并使研究人员能够将该工具套件用作更广泛的测量平台的一部分。Zmap工具已记录在案，可从https://zmap.io.下载生成的任何辅助数据集都将在https://scans.io.上发布ZMap计划在可预见的未来保持。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

It is difficult to predict Internet behavior due the complexity and distributed nature of the Internet. For example, it is hard to know whether proposed Internet protocols will work when deployed. As a result, researchers turn to large-scale, measurement of real-world systems to understand how the Internet is structured, how protocols operate, and where there are security and performance failures. This proposal improves a core set of measurement tools that researchers use for performing large-scale measurement of Internet hosts, websites, names, cloud assets to prevent common measurement errors, support recently introduced protocols, and account for real-world deployment complexities.This proposal will specifically make improvements to the ZMap suite of active measurement tools, including to (1) automatically detect error conditions and prevent erroneous data, (2) adapt tools to accommodate recently introduced network protocols like TLS 1.3, and (3) enable tools to find and accommodate services that appear in unpredictable locations. The challenge in these tasks lays in the fact that: (1) because there is no ground truth against which to compare results, tools need to intelligently determine whether results have become unstable and to alter their behavior while maintaining data consistency, (2) must operate with exceptional performance to accommodate the size of Internet-wide studies, and (3) must produce and structure data that researchers can use when research tasks are not known ahead of time.The impact of this project lies in the research that it will enable. ZMap has been used in many research publications over the past few years, but research has uncovered several shortcomings and error conditions that can produce inconsistent data. The Internet has also evolved over this time period, which requires software changes to accommodate. The improvements will help make ZMap accessible to additional researchers, ensure that researchers with less Internet measurement experience can reliably collect valid results, enable the study of recently introduced protocols, and enable researchers to use the suite of tools as part of broader measurement platforms.The ZMap tools are documented and available for download at https://zmap.io. Any auxiliary datasets produced will be published at https://scans.io. ZMap is planned to be maintained for the foreseeable future.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.