CAREER: Towards Efficient In-storage Indexing

职业：实现高效的存储内索引

• 批准号: 2338457
• 负责人: Janki Bhimani
• 金额: $ 61.55万
• 依托单位: Florida International University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-07-01 至 2029-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2338457&HistoricalAwards=false
• 关键词: CAREER; Towards; Efficient; storage; Indexing

Data indexing plays a crucial role in numerous modern technologies, including search engines, big data analytics, file systems, and databases. In this context, in-storage indexing devices (ISIDs) have emerged to enhance the functionalities of storage devices, leading to improved performance, efficiency, and cost-effective data processing. By storing index information alongside the data it indexes within the same storage device, ISIDs offer several advantages over traditional indexing methods. These advantages include reducing data movement, improving access speed, minimizing network impact, enabling efficient data management, and freeing host computing for critical tasks. To design efficient ISIDs, several challenges need to be addressed. Firstly, there is a need for low-cost and open-source research platforms to facilitate the reproduction and comparison of research work, promoting quick adoption of ISID advancements. Secondly, integrating the fragmented advancements of individual ISID components is crucial to capture their holistic impacts and interactions effectively. Thirdly, addressing diverse workload requests, interference in multi-tenant environments, and data distribution considerations requires new research methods for overall operation optimization. This CAREER research project aims to overcome these research challenges and promote the adoption of ISIDs, contributing to the advancements of storage systems. This project will explore and develop innovative methods to unleash the full potential of ISIDs in modern data management systems. By addressing the core challenges, the project seeks to revolutionize data storage systems and make significant contributions to the field of storage technology. This project will share the findings with undergraduate and graduate students through computer science programs and open up career opportunities to female students, underrepresented minorities, and first-generation college students. This project will disseminate the proposed techniques into the industry and foster technology transfer through new industrial collaborations. The developed infrastructure will be available to the research community through a web-based portal.This research makes significant empirical contributions to the ISID design and development space by addressing major challenges posed by in-storage indexing. Specifically, it advances the state of knowledge by investigating the following questions: (1) How can we design and develop new ISID models that accurately capture the behavior of internal modules, such as the index manager, request handler, data access parallelism, index-induced wear leveling, and garbage collection? These insights will enable scientific design advancements and detailed tradeoff analysis for ISIDs. (2) How can we develop efficient dynamic model calibration techniques using coarse measurements to parameterize queuing models that accurately capture burstiness and variability in ISIDs? (3) How can we emulate index manager delays using different data structures and sizes and utilize black-box and gray-box calibration techniques to identify ground truth for ISIDs? (4) How can we design a new re-configurable indexing architecture and index cache that ensures deterministic tail latency, low overhead prefetching and eviction, and improved membership checking through object signatures and ML-based feature learning in ISIDs? (5) How can we design tenant-local eviction policies that consider the effect of allocating space for index and data, recognizing the dependencies between them for efficient data access in ISIDs? (6) How can we minimize log-checking overhead and avoid in-storage hash computations while exploring the trade-off between consistency and performance by allowing read-only tenants to bypass the log and access their own consistent index in ISIDs? (7) Does capacity variance, which gracefully reduces ISID capacity as flash pages become bad, provide a better alternative to wear-leveling for ISIDs? Throughout the project, the PI will facilitate the connection of the proposed research with the contents and concepts of several courses on Systems at FIU.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据索引在许多现代技术中起着至关重要的作用，包括搜索引擎，大数据分析，文件系统和数据库。在这种情况下，已经出现了存储内部索引设备（ISID）以增强存储设备的功能，从而提高了性能，效率和具有成本效益的数据处理。通过将索引信息与其在同一存储设备中索引的数据一起存储，ISID提供了比传统索引方法的几个优点。这些优点包括降低数据移动，提高访问速度，最大程度地降低网络影响，实现有效的数据管理以及为关键任务释放主机计算。为了设计高效的ISID，需要解决一些挑战。首先，需要低成本和开源研究平台来促进研究工作的复制和比较，从而促进ISID的快速采用。其次，整合单个ISID组件的零散进步对于有效捕获其整体影响和相互作用至关重要。第三，解决各种工作量请求，在多租户环境中的干扰以及数据分配注意事项需要新的研究方法来进行整体操作优化。该职业研究项目旨在克服这些研究挑战并促进ISID的采用，从而有助于存储系统的进步。该项目将探索和开发创新的方法，以释放现代数据管理系统中ISID的全部潜力。通过应对核心挑战，该项目试图彻底改变数据存储系统，并为存储技术领域做出重大贡献。该项目将通过计算机科学课程与本科生和研究生分享研究结果，并为女学生，代表性不足的少数民族和第一代大学生提供职业机会。该项目将通过新的工业合作将所提出的技术传播到行业，并促进技术转移。开发的基础架构将通过基于网络的门户网站为研究社区提供。这项研究通过应对储存内部索引带来的重大挑战，为ISID设计和开发空间做出了重大的经验贡献。具体而言，它通过研究以下问题来提高知识状态：（1）我们如何设计和开发新的ISID模型，以准确捕获内部模块的行为，例如索引管理器，请求处理程序，数据访问处理程序，数据访问并行性，索引诱导的磨损水平以及垃圾收集？这些见解将使ISID的科学设计进步和详细的权衡分析。 （2）我们如何使用粗测量测量来开发有效的动态模型校准技术来参数化排队模型，以准确捕获ISID中的爆发性和可变性？ （3）我们如何使用不同的数据结构和大小模拟索引管理器延迟，并利用黑框和灰色框校准技术来识别ISID的地面真相？ （4）我们如何设计一个新的可重新配置索引架构和索引缓存，以确保确定性的尾部潜伏期，低架空预取和驱逐和驱逐，并通过对象签名和基于ML的功能学习在ISID中进行改进的会员资格检查吗？ （5）我们如何设计考虑为索引和数据分配空间的效果的租户 - 本地驱逐策略，以识别它们之间的依赖关系以在ISID中有效数据访问？ （6）如何通过允许仅阅读的租户绕过日志并访问ISID中的一致索引来探索一致性和绩效之间的权衡，从而最大程度地减少对日志检查开销的开销并避免存储空间计算？ （7）容量差异是否会随着闪存页面不良而优雅地降低ISID的容量，为ISID提供了更好的替代方案？在整个项目中，PI将促进拟议的研究与FIU系统上几门课程的内容和概念的联系。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子来评估的，并被认为是值得支持的。