SCI: II: The TeraFlow Project: High Performance Flows for Mining Large Distributed Data Archives

这项提议寻求开发用于从多个来源的大容量数据流的高性能流、合并、过滤和挖掘的技术。该应用空间的目标是斯隆数字巡天(SDSS)和国家虚拟天文台数据集。这项工作涉及开发、扩展现有技术，并应用协议和服务来扩展到现有的高速网络基础设施，以支持这些海量数据流的处理。该项目将把各种能力整合到SDSS教育项目Skyserver中。影响和智力优势在于拟议的工作直接启用SDSS和NVO数据集，并将同样适用于一组国土防御应用程序和其他科学领域数据。这项工作还将探索在跨高带宽延迟路径的大容量事务性备份场景中使用这些“teraflow”技术。

This proposal seeks to develop technology for the high performance streaming, merging, filtering and mining of high volume data streams from multiple sources. The application space is targeted at the Sloan Digital Sky Survey (SDSS) and National Virtual Observatory data sets. The work involves developing, extending existing technologies, and applying protocols and services to scale to existing high speed networking infrastructure to support the processing of these high volume data streams.The project will integrate capabilities into the SDSS education project, Skyserver. Impact and intellectual merit lie in the proposed work's direct enabling of SDSS and NVO data sets, and will apply just as well to a set of homeland defense applications and other science domain data. The work will also explore the use of these "teraflow" techniques in high volume transactional backup scenarios across high bandwidth-delay paths.