High Throughput I/O for Large Scale Data Repositories

适用于大规模数据存储库的高吞吐量 I/O

• 批准号: 0702728
• 负责人: Ali Tosun
• 金额: --
• 依托单位: University of Texas at San Antonio
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-05-01 至 2011-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0702728&HistoricalAwards=false
• 关键词: Throughput; Large; Scale; Data; Repositories

Many applications including geographical information systems, scientific visualization and igh-dimensional databases include large data repositories up to terabytes in size. Although terabytes of storage space is now achievable, efficient retrieval is a challenging problem. This research involves novel schemes to distribute data among parallel disks for efficient retrieval including schemes using replication, efficient retrieval algorithms and self-organizing schemes that adapt to disk failures, disk additions and changing query patterns.Declustering has attracted a lot of interest over the last few years and has applications in many areas including high-dimensional data management, geographical information systems and scientific visualization. Most of the declustering research have focused on spatial range queries and finding schemes with low worst-case additive error. This research investigates various aspects of declustering including novel declustering schemes, replicated declustering, heterogeneous declustering, adaptive declustering and declustering using multiple databases. The investigators approach every issue both theoretically and practically, study what is theoretically possible, what can be achieved in practice and try to close the gap between the two. The investigators study novel declustering schemes with solid theoretical foundations including number-theoretic declustering and design-theoretic declustering. Replication strategies for various types of queries including spatial range queries and arbitrary queries are studied. Retrieval algorithm for design-theoretic replication has linear complexity and guarantees worst-case retrieval cost. The investigators study tradeoffs in retrieval between complexity and retrieval cost and develop a suite of protocols for retrieval. This research involves adaptive declustering schemes that adapt to disk failures, disk additions and changing query types by moving buckets between disks during idle periods.

许多应用程序（包括地理信息系统、科学可视化和高维数据库）都包含高达 TB 大小的大型数据存储库。尽管现在可以实现 TB 级的存储空间，但高效检索仍然是一个具有挑战性的问题。 这项研究涉及在并行磁盘之间分配数据以实现高效检索的新颖方案，包括使用复制的方案、高效检索算法和适应磁盘故障、磁盘添加和更改查询模式的自组织方案。去簇在过去几年中引起了人们的广泛兴趣，并在包括高维数据管理、地理信息系统和科学可视化在内的许多领域都有应用。 大多数去聚类研究都集中在空间范围查询和寻找具有低最坏情况加性误差的方案。这项研究研究了分簇的各个方面，包括新颖的分簇方案、复制分簇、异构分簇、自适应分簇和使用多个数据库的分簇。研究者从理论上和实践上探讨每一个问题，研究什么是理论上可能的，什么是实践中可以实现的，并试图缩小两者之间的差距。研究人员研究了具有坚实理论基础的新颖分簇方案，包括数论分簇和设计理论分簇。研究了包括空间范围查询和任意查询在内的各种类型查询的复制策略。设计理论复制的检索算法具有线性复杂度并保证最坏情况下的检索成本。 研究人员研究了检索复杂性和检索成本之间的权衡，并开发了一套检索协议。这项研究涉及自适应分簇方案，该方案通过在空闲期间在磁盘之间移动存储桶来适应磁盘故障、磁盘添加和更改查询类型。