Collaborative Research: SDCI Sec: Metadata Management Software Tools to Support Research and Development of Cyberinfrastructure

对于网络社区来说，收集有代表性的互联网测量数据仍然是一个具有挑战性且往往难以实现的目标。障碍包括互联网的规模和范围，在捕获，过滤和采样高数据率的技术挑战，在一个分散的网络与根本分散的所有权，建设和操作仪器的成本，以及政治障碍获得测量的困难。 即使（或特别是）有所有这些障碍，对代表性互联网数据集的需求和重要性正在增加-这对严谨的科学互联网研究来说是个好消息。这一需求的主要驱动力是人们日益认识到各种重要和日益相互依赖的基础设施易受威胁，军备竞赛不断升级的一个主要限制因素是我们共享网络基础设施数据的方法仍然令人惊讶地原始。CAIDA已经开发了一个互联网测量数据目录-- IMDC --一个信息索引（元数据）关于数据集及其在各种使用策略下的可用性。 该目录解决了网络科学中的一个重大挑战：通过将可访问的互联网数据集的元数据组织到一个单一的存储库中来降低搜索数据的成本。该SDCI项目将支持整合迄今为止从IMDC开发、实施和使用中吸取的经验教训，以更好地支持网络安全研究和网络基础设施开发社区。 该项目的三个主要部署目标是：（1）通过简化的界面和工具，更容易地对相关数据进行索引、注释和导航，减轻提供数据者的负担;（2）从使用专有数据库后端（Oracle）转换为完全开放源码的解决方案;以及（3）通过研讨会，对安全相关数据集的新重视，和创建公共网络论坛来讨论数据共享问题。虽然该项目的重点是网络安全和网络基础设施研究社区，但我们提出的设计目标和推广计划明确针对一系列科学和工程社区。特别是，我们相信，拟议的软件开发可以支持和促进NSF新宣布的数据共享政策，从2011年1月起，该法案要求所有提案都包括一个关于研究人员如何与其他研究人员共享数据的计划。网络基础设施研究：最大限度地重复使用现有的互联网数据;减少收集冗余数据所花费的时间;减少开始新研究所需的工作量;促进分析的有效性和再现性以及网络基础设施研究方法的知识价值：最大限度地重复使用现有的互联网数据;减少收集冗余数据所花费的时间;减少开始新研究所需的工作量;促进分析和结果的有效性和可重复性;使互联网的纵向和跨学科研究成为可能;开辟变革性网络研究的新的跨领域领域。目录和相关研讨会的成功将促进互联网测量数据向学术、商业和政府部门的研究人员和安全专家的广泛传播。通过将面向教育的数据收集纳入目录，该项目在研究和教育之间建立了直接联系，并改善了计算机科学和工程领域代表性不足的群体对互联网研究的访问。 最重要的是，通过该项目创建的软件将帮助其他学科和部门开发自己的目录实例，以支持现在对NSF至关重要的数据管理计划类型。

Collecting representative Internet measurement data has remained a challenging and often elusive goal for the networking community. Obstacles include the Internet's scale and scope, technical challenges in capturing, filtering and sampling high data rates, difficulty obtaining measurements across a decentralized network with radically distributed ownership, cost of building and operating instrumentation, and political hurdles. Even (or especially) with all these obstacles, the demand for and importance of representative Internet data sets is increasing -- which is good news for rigorous scientific Internet research. The primary driver of this demand is the growing awareness of vulnerabilities to threats to various critical and increasingly interdependent infrastructures, and that a primary limiting factor in the escalating arms race is our surprisingly still primitive approach to sharing cyberinfrastructure data.CAIDA has developed an Internet Measurement Data Catalog -- IMDC -- an index of information (metadata) about data sets and their availability under various usage policies. The catalog addresses a significant challenge in network science: reducing the cost of searching for data by organizing metadata about accessible Internet data sets into a single repository.This SDCI project will support integration of lessons learned thus far from IMDC development, implementation, and usage to better support the cybersecurity research and cyberinfrastructure development communities. The project's three primary deployment goals are to: (1) reduce the burden on those contributing data via a streamlined interface and tools for easier indexing, annotation and navigation of relevant data; (2) convert from use of a proprietary database backend (Oracle) to a completely open source solution; and (3) expand the catalog's relevance to the cybersecurity and other research communities via workshops, new emphasis on security-relevant data sets, and creation of public web forums to discuss data-sharing issues.Although the focus for this project isutility for the cybersecurity and cyberinfrastructure research community, our proposed design objectives and outreach plans explicitly target a range of science and engineering communities.In particular, we believe the proposed software development can support and promote NSF's newly announced Data Sharing Policy, which as of January 2011 requires all proposals to include a plan for how researchers intend to share their data with other researchers.The intellectual merit of the proposed software development activities is a range of measurable benefits to cyberinfrastructure research: maximizing the re-use of existing Internet data; decreasing the time spent collecting redundant data; reducing the effort needed to start a new study; promoting validation and reproducibility of analyses and r intellectual merit of cyberinfrastructure research methods: maximizing the re-use of existing Internet data; decreasing the time spent collecting redundant data; reducing the effort needed to start a new study; promoting validation and reproducibility of analyses and results; enabling longitudinal and cross-disciplinary studies of the Internet; and opening up new cross-domain areas of transformative networking research.The broader impacts of this project are diverse. The success of the catalog and related workshops will facilitate wide dissemination of Internet measurement data to researchers and security experts across academic, commercial, and government sectors.By including education-oriented data collections in the catalog, this project creates an immediate link between research and education, and improves access to Internet research for underrepresented groups in computer science and engineering. Most importantly, the software created through this project will help other disciplines and sectors to develop their own catalog instances to support the type of data management plans now articulated as essential to NSF.