CCRI: Medium: FANTAIL -- facilitating advances in network topology analysis

互联网制图——对互联网拓扑结构进行科学的测量、分析和注释——已经成为计算机和网络科学的一个新领域。为了支持这一领域，全球多个互联网测量基础设施不断进行全面的拓扑映射测量实验。多年来，应用互联网数据分析中心（CAIDA）一直运营着这些测量基础设施平台（Archipelago），该平台自2007年9月以来一直支持全球互联网的科学测量实验。这些数据集已经在广泛的计算机科学分支学科中产生了影响。然而，对这类数据进行更有效的科学使用的最大障碍是基础设施：缺乏易于使用和分析功能强大的探索界面。该项目将开发FANTAIL系统（促进网络拓扑分析的进步），以实现大量原始互联网端到端路径测量数据集的全部潜在价值。提出的系统将实现新的研究方向，并增加我们对互联网作为关键基础设施的理解。CAIDA的解决方案将降低进入网络科学和计算研究领域的门槛，从而增加用户基础的多样性。FANTAIL将有四个组成部分：(1)交互式web界面；(2)基于web标准的应用程序编程接口（API）；(3)全文检索系统；(4)大数据处理系统。该项目将由特定的社区请求用例驱动，在可能的情况下使用开源组件，并创建分析模块，这些模块将作为构建更复杂的数据处理管道的原语。用户将通过web界面或API指定一系列分析模块，以便在与其查询匹配的traceroute路径集上执行。然后，FANTAIL系统将执行查询，运行分析模块，并提供输出供下载，供研究人员在自己的系统上进一步分析或处理。不同的分析模块将(1)执行数据缩减（以最大限度地减少用户需要下载和处理的数据量），(2)使用公开可用或由CAIDA创建的各种注释增强原始拓扑数据，以及(3)卸载用户通常需要的分析/数据处理任务。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Internet cartography--the scientific measurement, analysis, and annotation of Internet topological structure--has emerged as a new field in computer and network science. In support of this field, several global Internet measurement infrastructures continuously execute comprehensive topology mapping measurement experiments. For years the Center for Applied Internet Data Analysis (CAIDA) has operated the longest-running of these measurement infrastructure platforms (Archipelago), which has supported scientific measurement experiments of the global Internet since September 2007. These data sets have already yielded impacts across a broad range of computer science sub-disciplines. Yet the biggest remaining obstacle to even more productive scientific use of this type of data is infrastructural: the lack of an easy-to-use and analytically powerful exploratory interface. This project will develop the FANTAIL system--Facilitating Advances in Network Topology Analysis--to enable the full potential value of massive raw Internet end-to-end path measurement data sets. The proposed system will enable new research directions and increase our understanding of the Internet as critical infrastructure. CAIDA's solutions will lower barriers for entering the field of network science and computing research thus leading to increased diversity of the user base.FANTAIL will have four components: (1) an interactive web interface; (2) an application programming interface (API) built on web standards; (3) a full-text search system; and (4) a big-data processing system. This project will be driven by specific community-requested use cases, use open-source components where possible, and create analysis modules that will serve as primitives for constructing more complex data-processing pipelines. Users will specify, via the web interface or API, a sequence of analysis modules to execute on the set of traceroute paths matched by their queries. The FANTAIL system will then perform the queries, run the analysis modules, and provide the output for download for further analysis or processing by researchers on their own systems. Different analysis modules will (1) perform data reduction (to minimize the amount of data users need to download and process), (2) enhance raw topology data with various annotations available publicly or created by CAIDA, and (3) offload commonly-needed analysis/data processing tasks from users.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.