A network-aware distributed storage cache for data intensive environments
A network-aware distributed storage cache for data intensive environments
复制标题
DOI：
10.1109/hpdc.1999.805297
复制
发表时间：
1999-08
期刊：
影响因子：
--
通讯作者：
B. Tierney;Jason R. Lee;B. Crowley;M. Holding;J. Hylton;F. Drake
中科院分区：
文献类型：
--
作者：
B. Tierney;Jason R. Lee;B. Crowley;M. Holding;J. Hylton;F. Drake
现代科学计算涉及组织、移动、可视化和分析世界各地多个站点的大量数据。用于构建有用的高速广域分布式系统的技术、中间件服务和体系结构构成了数据密集型计算领域。我们描述了一种用于数据密集型应用程序的体系结构，其中我们使用高速分布式数据缓存作为所有数据源和数据接收器的公共元素。这种基于缓存的方法为大型的、面向应用的、分布式的、在线的、瞬时的存储系统提供了标准接口。我们描述了这个缓存的实现，我们如何使它“网络感知”，以及我们如何基于当前的网络条件进行动态负载平衡。通过访问网络条件的知识，我们还显示了应用程序吞吐量的大幅增加。
Modern scientific computing involves organizing, moving, visualizing, and analyzing massive amounts of data at multiple sites around the world. The technologies, the middleware services, and the architectures that are used to build useful high-speed, wide area distributed systems, constitute the field of data intensive computing. We describe an architecture for data intensive applications where we use a high-speed distributed data cache as a common element for all of the sources and sinks of data. This cache-based approach provides standard interfaces to a large, application-oriented, distributed, on-line, transient storage system. We describe our implementation of this cache, how we have made it "network aware ", and how we do dynamic load balancing based on the current network conditions. We also show large increases in application throughput by access to knowledge of the network conditions.