The Terabyte Challenge: Developing Software Tools and Network Services for Mining Remote and Distributed Data Over High Performance Networks

随着下一代高性能网络的部署，以及在互联网上移动大量数据变得可行，科学家和工程师将首次有能力以随意的方式分析和挖掘远程和分布式数据。数据挖掘中的半自动发现模式、关联、变化和异常是大型数据集。传统上，从广义上讲，统计学侧重于数据的假设驱动分析，而数据挖掘侧重于数据的发现分析。通过发现驱动，提议者指的是对有趣模式和模型的半自动搜索。该项目将开发软件工具、性能监控工具、网络服务和网络协议，用于使用新兴的高性能网络进行分布式数据分析和数据挖掘。项目主管建议使用正在建造的称为tb挑战测试平台（TCTB）的全球测试平台来测试开发的技术。今天，来自三大洲七个机构的二十多名科学家正在TCTB工作。项目主管建议通过此提案大幅扩展TCTB，以展示分布式数据挖掘的实用性。

As next generation high performance networks are deployed and it becomes practical to move large amounts of data over the internet, scientists and engineers will have the ability for the first time to analyze and mine remote and distributed data in casual way.Data mining in the semi-automatic discovery of patterns, associations, changes, and anomalies is large data sets. Traditionally, in a broad sense, statistics had focused on the assumption-driven analysis of data, while data mining has focused on the discovery analysis of data. By discovery-driven, the proposers mean the semi-automatic search for interesting patterns and models.This project will develop software tools, performance monitoring tools, network services and network protocols for distributed data analysis and data mining using emerging high performance networks. The project director proposes to test the technologies that are developed using a global testbed being built called the Terabyte Challenge Testbed (TCTB). Today over twenty scientists from seven institutions over three continents are working in the TCTB. The project director proposes to substantially expand the TCTB with this proposal to demonstrate the practically of distributed data mining.