Planning a Globally Assessible Archive of MODIS Data

基于卫星的仪器，如NASA的中分辨率成像光谱仪（MODIS）系统，产生了大量独特的数据，这给必须管理这些数据的数据和档案系统带来了一个普遍的问题。每天大约有1TB的流量（4年接近2pb）， MODIS数据的高价值要求它被可靠地保存在长期档案存储中，但与此同时，它目前的国际用户社区希望它能在广域范围内保持持续可用。大量数据量（其中大部分必须存储在线路附近）和高度分散的用户群（其中一些用户需要临时访问甚至交互式访问）的结合，加剧了深度档案存储和广域网之间长期存在的技术和管理不匹配。这个计划项目将把MODIS档案社区的领导者与计算机科学研究人员聚集在一起，开创广域数据管理的新方法。MAAP项目的目的是计划一个可资助的分布式服务平台的试验演示，为全球研究、教育和政策规划社区提供快速、灵活和有效地访问所有MODIS存档数据。为了应对在网络环境中管理海量数据的后勤挑战，该项目将探索一种新的通用存储服务，即互联网背板协议（IBP），它应用互联网设计理念来创建一种可以随网络扩展的网络服务。该技术已经在经过良好测试的中间件集合中实现，并已部署在具有400多个节点和40TB存储的全球测试平台中。各种应用社区，包括大规模内容分发、分布式数据挖掘和成像数据的远程可视化已经在使用该技术。为了创建一个成功的MAAP演示，该团队将直接与NASA-USGS陆地过程分布式主动档案中心（LP-DAAC）的科学和管理团队合作。

The ongoing flood of unique data generated by satellite based instruments, such as NASA's Moderate Resolution Imaging Spectroradiometer (MODIS) systems, creates a general problem for data and archival systems that must manage it. Flowing in at roughly 1TB a day (approaching 2 PB over 4 years), MODIS data's high value requires that it be reliably preserved in long-term archival storage, yet at the same time, its current international user community wants it to remain continuously available for wide area access. The combination of massive data volumes, most of which must be stored near line, and a highly distributed user base, some of whom demand ad hoc or even interactive access, exacerbates the long standing technical and managerial mismatch between deep archival storage and wide area networks. This planning project will bring together leaders of the MODIS archive community with Computer Science researchers pioneering a new approach to wide area data management. The purpose of the MAAP project is to plan a fundable trial demonstration of a distributed service platform for providing fast, flexible, and efficient access to all MODIS archive data for a global research, education and policy planning community. In order to deal with the logistical challenges of managing massive data in networked environments, this project will explore a new common storage service, the Internet Backplane Protocol (IBP), which applies the Internet design philosophy to create a network service that can scale with the network. This technology has already been implemented in a well tested collection of middleware and has been deployed in a global testbed with over 400 nodes and 40TB of storage. Avariety of application communities, including large scale content distribution, distributed data mining, andremote visualization of imaging data already use this technology. To create a successful MAAP demonstration, the team will directly engage with the scientific and management teams of the NASA-USGS Land Processes Distributed Active Archive Center (LP-DAAC).