Collaborative Research: SDCI Sec: Metadata Management Software Tools to Support Research and Development of Cyberinfrastructure

对于网络社区来说，收集有代表性的互联网测量数据一直是一个具有挑战性和难以捉摸的目标。障碍包括互联网的规模和范围，在捕获、过滤和采样高数据速率方面的技术挑战，在一个分散的网络上获得测量数据的困难，所有权完全分散，构建和操作仪器的成本，以及政治障碍。即使（或特别是）存在所有这些障碍，对具有代表性的互联网数据集的需求和重要性正在增加——这对严格的科学互联网研究来说是个好消息。这一需求的主要驱动因素是，人们越来越意识到各种关键和日益相互依赖的基础设施面临的威胁，而军备竞赛不断升级的一个主要限制因素是，我们共享网络基础设施数据的方式令人惊讶地仍然很原始。CAIDA开发了一个互联网测量数据目录（IMDC），这是一个关于数据集及其在各种使用政策下的可用性的信息索引（元数据）。该目录解决了网络科学中的一个重大挑战：通过将有关可访问的Internet数据集的元数据组织到单个存储库中来降低搜索数据的成本。该SDCI项目将支持整合迄今为止从IMDC开发、实施和使用中吸取的经验教训，以更好地支持网络安全研究和网络基础设施开发社区。该项目的三个主要部署目标是：(1)通过简化的界面和工具减轻提供数据的人的负担，从而更容易对相关数据进行索引、注释和导航；(2)从使用专有数据库后端（Oracle）转变为完全开源的解决方案；(3)通过举办研讨会、强调与安全相关的数据集以及创建公共网络论坛来讨论数据共享问题，扩大目录与网络安全和其他研究社区的相关性。虽然这个项目的重点是网络安全和网络基础设施研究界的实用性，但我们提出的设计目标和推广计划明确针对一系列科学和工程社区。特别是，我们相信拟议的软件开发可以支持和促进NSF新宣布的数据共享政策，该政策自2011年1月起要求所有提案包括研究人员打算如何与其他研究人员共享数据的计划。建议的软件开发活动的智力价值是对网络基础设施研究的一系列可衡量的好处：最大限度地重用现有的互联网数据；减少收集冗余数据所花费的时间；减少开始一项新研究所需的努力；促进网络基础设施研究方法的分析和智力价值的验证和可重复性：最大限度地重复利用现有互联网数据；减少收集冗余数据所花费的时间；减少开始一项新研究所需的努力；促进分析和结果的验证和可重复性；促进互联网的纵向和跨学科研究；并开辟新的跨领域的变革性网络研究领域。这个项目的广泛影响是多方面的。目录和相关研讨会的成功将促进互联网测量数据向学术、商业和政府部门的研究人员和安全专家的广泛传播。通过在目录中包括以教育为导向的数据收集，该项目在研究和教育之间建立了直接的联系，并改善了计算机科学和工程领域代表性不足的群体对互联网研究的访问。最重要的是，通过这个项目创建的软件将帮助其他学科和部门开发他们自己的目录实例，以支持现在对NSF至关重要的数据管理计划类型。

Collecting representative Internet measurement data has remained a challenging and often elusive goal for the networking community. Obstacles include the Internet's scale and scope, technical challenges in capturing, filtering and sampling high data rates, difficulty obtaining measurements across a decentralized network with radically distributed ownership, cost of building and operating instrumentation, and political hurdles. Even (or especially) with all these obstacles, the demand for and importance of representative Internet data sets is increasing -- which is good news for rigorous scientific Internet research. The primary driver of this demand is the growing awareness of vulnerabilities to threats to various critical and increasingly interdependent infrastructures, and that a primary limiting factor in the escalating arms race is our surprisingly still primitive approach to sharing cyberinfrastructure data.CAIDA has developed an Internet Measurement Data Catalog -- IMDC -- an index of information (metadata) about data sets and their availability under various usage policies. The catalog addresses a significant challenge in network science: reducing the cost of searching for data by organizing metadata about accessible Internet data sets into a single repository.This SDCI project will support integration of lessons learned thus far from IMDC development, implementation, and usage to better support the cybersecurity research and cyberinfrastructure development communities. The project's three primary deployment goals are to: (1) reduce the burden on those contributing data via a streamlined interface and tools for easier indexing, annotation and navigation of relevant data; (2) convert from use of a proprietary database backend (Oracle) to a completely open source solution; and (3) expand the catalog's relevance to the cybersecurity and other research communities via workshops, new emphasis on security-relevant data sets, and creation of public web forums to discuss data-sharing issues.Although the focus for this project isutility for the cybersecurity and cyberinfrastructure research community, our proposed design objectives and outreach plans explicitly target a range of science and engineering communities.In particular, we believe the proposed software development can support and promote NSF's newly announced Data Sharing Policy, which as of January 2011 requires all proposals to include a plan for how researchers intend to share their data with other researchers.The intellectual merit of the proposed software development activities is a range of measurable benefits to cyberinfrastructure research: maximizing the re-use of existing Internet data; decreasing the time spent collecting redundant data; reducing the effort needed to start a new study; promoting validation and reproducibility of analyses and r intellectual merit of cyberinfrastructure research methods: maximizing the re-use of existing Internet data; decreasing the time spent collecting redundant data; reducing the effort needed to start a new study; promoting validation and reproducibility of analyses and results; enabling longitudinal and cross-disciplinary studies of the Internet; and opening up new cross-domain areas of transformative networking research.The broader impacts of this project are diverse. The success of the catalog and related workshops will facilitate wide dissemination of Internet measurement data to researchers and security experts across academic, commercial, and government sectors.By including education-oriented data collections in the catalog, this project creates an immediate link between research and education, and improves access to Internet research for underrepresented groups in computer science and engineering. Most importantly, the software created through this project will help other disciplines and sectors to develop their own catalog instances to support the type of data management plans now articulated as essential to NSF.