CSR: Small: Managing and Indexing Exascale Archival Storage Systems

该奖项是根据2009年《美国复苏和再投资法案》(Public Law 111-5)资助的。随着我们的社会存储越来越多的数字数据并放弃模拟存储，我们必须能够管理、组织和保存数字数据数十年或更长时间。保持对大规模数字档案的访问对于使后代能够访问我们正在生成和存储的数字医疗记录、个人信息、照片和其他数据将是至关重要的。然而，当前的档案存储方法不适合长期保存，因为它们不能很好地应对设备技术的发展，并且依赖于不能很好扩展且容易失败的集中式搜索索引。该项目正在探索通过网络连接的包含105？106个智能电源管理存储设备的大规模档案管理技术。这些技术通过集成新设备并移除旧的低效设备，实现了档案存储系统的无缝发展。该研究还在开发为档案中的数据编制索引的方法，允许用户通过在每个设备上维护索引并将查询发送到可能包含相关数据的设备来快速找到他们需要的数据。这些技术将使档案可以扩展到数十万台设备，使它们能够包含我们留给后代的大量数字遗产。这项研究将指导设计高效的档案，并可以随着存储技术的变化而优雅地发展。此外，该项目将对本科生和研究生进行关于数字档案面临的问题的培训，这是一个至关重要的领域。

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).As our society stores ever-increasing quantities of digital data and abandons analog storage, we must be able to manage, organize, and preserve digital data for decades or longer. Maintaining access to large-scale digital archives will be critical in enabling future generations to access the medical records, personal information, photographs, and other data that we are generating and storing digitally. However, current approaches to archival storage are ill-suited to long-term preservation because they do not cope well with evolution of device technologies and rely upon centralized search indexes that do not scale well and are prone to failure.This project is exploring techniques that allow the management of large-scale archives containing 105?106 intelligent power-managed storage devices connected by a network. These techniques allow seamless evolution of the archival storage system by integrating new devices and removing old, inefficient devices. The research is also developing approaches to index data in the archive, allowing users to quickly find the data they need by maintaining indexes on each device and routing queries to the devices that might contain relevant data. These techniques will allow archives to scale to hundreds of thousands of devices, allowing them to contain the vast digital legacy we are leaving to our descendants.This research will guide the design of archives that are power-efficient and can gracefully evolve as storage technology changes. Additionally, the project will train both undergraduates and graduate students in the problems facing digital archiving, an area of critical importance.