ITR/SY: Algorithms for Data Storage and Movement

该建议的中心焦点是开发用于存储和移动数据的有效算法。具体来说，我们对影响大型多媒体数据存储系统性能的算法感兴趣。在算法方面，多媒体数据存储环境中出现的一些主要挑战是：(a)决定需要存储每个数据项的多少份副本，(b)确定一组服务器上数据的确切布局，(c)处理不断变化的工作负载和动态数据访问模式。这些相关的挑战需要开发有效的算法来优化数据布局，以最大限度地提高客户满意度，监控数据存储系统的性能，并安排大量数据的移动。此外，使我们考虑的问题更加重要的是，数据存储和移动问题也出现在公共共享网络（如互联网）中，其中带宽可能是动态和高度可变的，并且可能导致在网络中传输数据的路径选择不佳。解决这个问题的一种方法是通过特定的保存点路由数据。通过这样做，我们能够提高吞吐量并将完成时间减少一个数量级，从而将数据从多个源传输到单个目的地。我们已经开发了与这个问题相关的算法，并正在用Bistro框架进行测试，Bistro框架是一个提供数据上传服务的框架，例如IRS为提交税款而要求的数据上传服务。我们的数据移动算法被用于调度数据从许多不同位置到最终目标服务器的传输。虽然早些时候已经审议了个别问题的一些具体情况，但没有工作全面处理我们所关注的一系列问题。

The central focus of this proposal is the development of efficient algorithms for the storage and movement of data. Specifically, we are interested in algorithms that impact the performance of large multimedia data storage systems. In algorithmic terms, some of the principal challenges that arise in the context of multimedia data storage are: (a) deciding how many copies of each data item need to be stored, (b) determining the exact layout of data on a set of servers, (c) dealing with changing workloads and dynamic data access patterns. These related challenges require the development of efficient algorithms for optimizing data layout to maximize client satisfaction, monitoring the performance of data storage systems and scheduling the movement of large amounts of data.Futhermore, what makes the issues that we consider even more significant is the fact that data storage and movement issues also arise within publicly share networks such as the Internet where the bandwidth can be dynamic and highly variable, and can result in a poor choice of paths chosen to transfer data in the network. One way to address this issue is to route data through specific holding points. By doing this we are able to increase throughput and decrease completion times by an order of magnitude to transfer data from several sources to a single destination. Algorithms related to this problem have been developed by us and are being tested with the Bistro framework, which is a framework for providing a data upload service such as one required by IRS for tax submission purposes. Our data movement algorithms are being used to schedule the transfer of data from many different locations to a final destination server.While some specific instances of the individual problems have been considered earlier, there is no work dealing comprehensively with the range of issues that we focus on.