CSR-PDOS-Content-Searchable Storage for Feature-Rich Data

点击翻译按钮获取中文摘要

Storage capacity and data volume have been doubling every 18 months during the pasttwo decades. A key challenging issue in building next-generation storage systems is tomanage massive amounts of feature-rich (non-text) data, which has dominated theincreasing volume of digital information. Comparing noisy, feature-rich data requires fast similarity match instead of exact match, and thus exploring such data requires similaritysearch instead of exact search. Current file systems are designed for named text files;they do not have mechanisms to manage feature-rich data. To date, there is no practicalstorage system with the ability to do similarity search for noisy, high-dimensional dataand there is no index engine design for efficient similarity search. This researchaddresses this problem by studying how to design and implement a content-addressableand -searchable storage (CASS) system to manage and explore diverse feature-rich data.The system includes a built-in similarity search engine for general-purpose, noisy, highdimensionalmetadata using compact data structures and novel indexing methods. Theresearch will also develop segmentation methods and feature extraction methods foraudio, image and genomic data, and develop similarity search benchmarks and toevaluate the CASS system.This research will advance knowledge and understanding in the area of storage systemdesigns such as data structures, mechanisms, and APIs for managing, searching andexploring noisy, high-dimensional feature-rich data. The research will accelerate thedevelopment of next-generation storage systems which will revolutionize how to access,search, explore and manage massive amounts of feature-rich data in many disciplines.