CCRI: Medium: FANTAIL -- facilitating advances in network topology analysis

互联网地图学--对互联网拓扑结构的科学测量、分析和注释--已经成为计算机和网络科学中的一个新领域。为支持这一领域，全球多个互联网测量基础设施不断进行全面的拓扑映射测量实验。多年来，应用互联网数据分析中心(CAIDA)一直运营着这些测量基础设施平台中运行时间最长的(群岛)，自2007年9月以来一直支持全球互联网的科学测量实验。这些数据集已经在广泛的计算机科学分支学科中产生了影响。然而，要更有效地科学利用这种类型的数据，剩下的最大障碍是基础设施：缺乏易于使用和分析功能强大的探索界面。该项目将开发FanTail系统--促进网络拓扑分析的进步--以充分发挥海量原始互联网端到端路径测量数据集的潜在价值。拟议的系统将提供新的研究方向，并增加我们对互联网作为关键基础设施的理解。CAIDA的解决方案将降低进入网络科学和计算研究领域的门槛，从而增加用户基础的多样性。FANTAIL将有四个组成部分：(1)交互式网络界面；(2)基于网络标准的应用编程接口；(3)全文搜索系统；和(4)大数据处理系统。该项目将由社区要求的特定用例驱动，尽可能使用开放源码组件，并创建分析模块，作为构建更复杂的数据处理管道的原语。用户将通过Web界面或API指定一系列分析模块，以在与其查询匹配的一组Traceroute路径上执行。然后，扇尾系统将执行查询，运行分析模块，并提供输出供下载，供研究人员在自己的系统上进一步分析或处理。不同的分析模块将(1)执行数据简化(以最大限度地减少用户需要下载和处理的数据量)，(2)使用公开提供的或由CAIDA创建的各种注释来增强原始拓扑数据，以及(3)从用户那里卸载常用的分析/数据处理任务。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Internet cartography--the scientific measurement, analysis, and annotation of Internet topological structure--has emerged as a new field in computer and network science. In support of this field, several global Internet measurement infrastructures continuously execute comprehensive topology mapping measurement experiments. For years the Center for Applied Internet Data Analysis (CAIDA) has operated the longest-running of these measurement infrastructure platforms (Archipelago), which has supported scientific measurement experiments of the global Internet since September 2007. These data sets have already yielded impacts across a broad range of computer science sub-disciplines. Yet the biggest remaining obstacle to even more productive scientific use of this type of data is infrastructural: the lack of an easy-to-use and analytically powerful exploratory interface. This project will develop the FANTAIL system--Facilitating Advances in Network Topology Analysis--to enable the full potential value of massive raw Internet end-to-end path measurement data sets. The proposed system will enable new research directions and increase our understanding of the Internet as critical infrastructure. CAIDA's solutions will lower barriers for entering the field of network science and computing research thus leading to increased diversity of the user base.FANTAIL will have four components: (1) an interactive web interface; (2) an application programming interface (API) built on web standards; (3) a full-text search system; and (4) a big-data processing system. This project will be driven by specific community-requested use cases, use open-source components where possible, and create analysis modules that will serve as primitives for constructing more complex data-processing pipelines. Users will specify, via the web interface or API, a sequence of analysis modules to execute on the set of traceroute paths matched by their queries. The FANTAIL system will then perform the queries, run the analysis modules, and provide the output for download for further analysis or processing by researchers on their own systems. Different analysis modules will (1) perform data reduction (to minimize the amount of data users need to download and process), (2) enhance raw topology data with various annotations available publicly or created by CAIDA, and (3) offload commonly-needed analysis/data processing tasks from users.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.