CAREER: Towards Efficient In-storage Indexing

数据索引在许多现代技术中起着至关重要的作用，包括搜索引擎、大数据分析、文件系统和数据库。在这种情况下，存储索引设备（ISID）已经出现，以增强存储设备的功能，从而提高性能，效率和成本效益的数据处理。通过将索引信息与索引数据一起存储在同一存储设备中，ISID提供了几个优于传统索引方法的优点。这些优势包括减少数据移动，提高访问速度，最大限度地减少网络影响，实现高效的数据管理，并释放主机计算以执行关键任务。为了设计高效的ISID，需要解决几个挑战。首先，需要低成本和开源的研究平台，以促进研究工作的复制和比较，促进快速采用ISID的进步。其次，整合各个包容及可持续工业发展组成部分的零散进展对于有效捕捉其整体影响和互动至关重要。第三，解决不同的工作负载请求、多租户环境中的干扰和数据分布考虑因素，需要新的研究方法来实现整体运营优化。这个CAREER研究项目旨在克服这些研究挑战，促进ISID的采用，为存储系统的进步做出贡献。该项目将探索和开发创新方法，在现代数据管理系统中充分发挥ISIDs的潜力。通过解决核心挑战，该项目旨在彻底改变数据存储系统，并为存储技术领域做出重大贡献。该项目将通过计算机科学课程与本科生和研究生分享研究结果，并为女学生，代表性不足的少数民族和第一代大学生提供就业机会。该项目将把拟议的技术传播到工业界，并通过新的工业合作促进技术转让。开发的基础设施将通过一个基于网络的门户网站提供给研究界，这项研究通过解决存储索引所带来的主要挑战，为ISID的设计和开发空间做出了重大的经验贡献。具体来说，它通过调查以下问题来提高知识水平：（1）我们如何设计和开发新的ISID模型，以准确地捕获内部模块的行为，如索引管理器，请求处理程序，数据访问并行性，索引引起的磨损均衡和垃圾收集？这些见解将使ISID的科学设计进步和详细的权衡分析成为可能。(2)我们如何开发有效的动态模型校准技术，使用粗略的测量来参数化排队模型，以准确地捕捉ISID中的突发性和可变性？(3)我们如何使用不同的数据结构和大小来模拟索引管理器延迟，并利用黑盒和灰盒校准技术来识别ISID的基本事实？(4)我们如何设计一个新的可重新配置的索引架构和索引缓存，以确保确定性的尾部延迟，低开销的预取和驱逐，并通过ISID中的对象签名和基于ML的特征学习改进成员资格检查？(5)我们如何设计租户本地驱逐策略，考虑为索引和数据分配空间的影响，识别它们之间的依赖关系，以实现ISID中的高效数据访问？(6)我们如何通过允许只读租户绕过日志并访问ISID中自己的一致性索引来最大限度地减少日志检查开销并避免存储中的哈希计算，同时探索一致性和性能之间的权衡？(7)容量差异（随着闪存页面变坏而优雅地减少ISID容量）是否为ISID的损耗均衡提供了更好的替代方案？在整个项目中，PI将促进拟议研究与FIU系统课程的内容和概念的联系。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。

Data indexing plays a crucial role in numerous modern technologies, including search engines, big data analytics, file systems, and databases. In this context, in-storage indexing devices (ISIDs) have emerged to enhance the functionalities of storage devices, leading to improved performance, efficiency, and cost-effective data processing. By storing index information alongside the data it indexes within the same storage device, ISIDs offer several advantages over traditional indexing methods. These advantages include reducing data movement, improving access speed, minimizing network impact, enabling efficient data management, and freeing host computing for critical tasks. To design efficient ISIDs, several challenges need to be addressed. Firstly, there is a need for low-cost and open-source research platforms to facilitate the reproduction and comparison of research work, promoting quick adoption of ISID advancements. Secondly, integrating the fragmented advancements of individual ISID components is crucial to capture their holistic impacts and interactions effectively. Thirdly, addressing diverse workload requests, interference in multi-tenant environments, and data distribution considerations requires new research methods for overall operation optimization. This CAREER research project aims to overcome these research challenges and promote the adoption of ISIDs, contributing to the advancements of storage systems. This project will explore and develop innovative methods to unleash the full potential of ISIDs in modern data management systems. By addressing the core challenges, the project seeks to revolutionize data storage systems and make significant contributions to the field of storage technology. This project will share the findings with undergraduate and graduate students through computer science programs and open up career opportunities to female students, underrepresented minorities, and first-generation college students. This project will disseminate the proposed techniques into the industry and foster technology transfer through new industrial collaborations. The developed infrastructure will be available to the research community through a web-based portal.This research makes significant empirical contributions to the ISID design and development space by addressing major challenges posed by in-storage indexing. Specifically, it advances the state of knowledge by investigating the following questions: (1) How can we design and develop new ISID models that accurately capture the behavior of internal modules, such as the index manager, request handler, data access parallelism, index-induced wear leveling, and garbage collection? These insights will enable scientific design advancements and detailed tradeoff analysis for ISIDs. (2) How can we develop efficient dynamic model calibration techniques using coarse measurements to parameterize queuing models that accurately capture burstiness and variability in ISIDs? (3) How can we emulate index manager delays using different data structures and sizes and utilize black-box and gray-box calibration techniques to identify ground truth for ISIDs? (4) How can we design a new re-configurable indexing architecture and index cache that ensures deterministic tail latency, low overhead prefetching and eviction, and improved membership checking through object signatures and ML-based feature learning in ISIDs? (5) How can we design tenant-local eviction policies that consider the effect of allocating space for index and data, recognizing the dependencies between them for efficient data access in ISIDs? (6) How can we minimize log-checking overhead and avoid in-storage hash computations while exploring the trade-off between consistency and performance by allowing read-only tenants to bypass the log and access their own consistent index in ISIDs? (7) Does capacity variance, which gracefully reduces ISID capacity as flash pages become bad, provide a better alternative to wear-leveling for ISIDs? Throughout the project, the PI will facilitate the connection of the proposed research with the contents and concepts of several courses on Systems at FIU.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.