Planning a Globally Assessible Archive of MODIS Data

由卫星仪器(如NASA的中分辨率成像光谱仪(MODIS)系统)产生的不断涌现的独特数据，给必须管理这些数据的数据和档案系统带来了普遍的问题。MODIS数据以每天约1TB的速度流动(在4年内接近2PB)，其高价值要求它能够可靠地保存在长期的档案存储中，但与此同时，其当前的国际用户群体希望它保持持续可用，以供广域访问。海量数据(其中大部分必须存储在网上)和高度分散的用户群(其中一些人需要临时甚至交互访问)相结合，加剧了深度档案存储和广域网之间长期存在的技术和管理不匹配。这个规划项目将把MODIS档案界的领导人和计算机科学研究人员聚集在一起，开创一种新的广域数据管理方法。MAAP项目的目的是计划一个分布式服务平台的可资助试验演示，为全球研究、教育和政策规划社区提供对所有MODIS档案数据的快速、灵活和高效的访问。为了应对网络环境中管理海量数据的后勤挑战，该项目将探索一种新的通用存储服务--互联网背板协议(IBP)，该协议应用互联网设计理念来创建可随网络扩展的网络服务。这项技术已经在经过良好测试的中间件集合中实施，并已部署在拥有400多个节点和40TB存储的全球试验台中。各种应用社区，包括大规模的内容分发、分布式数据挖掘和图像数据的远程可视化，都已经在使用该技术。为了创建一个成功的MAAP演示，该团队将直接与NASA-USGS陆地过程分布式活动档案中心(LP-DAAC)的科学和管理团队接触。

The ongoing flood of unique data generated by satellite based instruments, such as NASA's Moderate Resolution Imaging Spectroradiometer (MODIS) systems, creates a general problem for data and archival systems that must manage it. Flowing in at roughly 1TB a day (approaching 2 PB over 4 years), MODIS data's high value requires that it be reliably preserved in long-term archival storage, yet at the same time, its current international user community wants it to remain continuously available for wide area access. The combination of massive data volumes, most of which must be stored near line, and a highly distributed user base, some of whom demand ad hoc or even interactive access, exacerbates the long standing technical and managerial mismatch between deep archival storage and wide area networks. This planning project will bring together leaders of the MODIS archive community with Computer Science researchers pioneering a new approach to wide area data management. The purpose of the MAAP project is to plan a fundable trial demonstration of a distributed service platform for providing fast, flexible, and efficient access to all MODIS archive data for a global research, education and policy planning community. In order to deal with the logistical challenges of managing massive data in networked environments, this project will explore a new common storage service, the Internet Backplane Protocol (IBP), which applies the Internet design philosophy to create a network service that can scale with the network. This technology has already been implemented in a well tested collection of middleware and has been deployed in a global testbed with over 400 nodes and 40TB of storage. Avariety of application communities, including large scale content distribution, distributed data mining, andremote visualization of imaging data already use this technology. To create a successful MAAP demonstration, the team will directly engage with the scientific and management teams of the NASA-USGS Land Processes Distributed Active Archive Center (LP-DAAC).