CRII: III: Scalable Noise-filtering and Community Queries on User-generated Data

该项目研究新的索引和查询技术，使科学家能够分析并从用户生成的大型数据存储库中提取有意义的数据。对这种技术的需求很大，特别是对用户生成的社交媒体数据，这是用户生成数据的主要存储库，也是人类行为和信息的最大存档和实时来源。因此，科学家在社会学、行为科学、教育、空间科学、食品科学、医学研究和政治学等不同学科中广泛使用这些数据。该项目专注于创新的索引和查询技术，使科学家能够大规模有效地利用用户生成的数据。计划中的研究增加了新的数据管理基础设施模块，以支持：(1)可伸缩噪声过滤查询，这是选择查询的子集，重复需要，在基于SQL的系统中表示为多个嵌套查询，这对于大型数据集来说效率不高。为支持这一点，该项目研究了以下技术：(A)高级查询连接，例如，但不与或异或，以在数据管理系统目前支持的基本搜索查询之外扩大复杂谓词查询。(B)数据记录的可扩展的上下文评分，例如基于情绪或语义，因此不相关的记录被及早修剪，并且搜索空间被显著缩小。(2)可扩展的以社区为中心的查询，使科学家能够询问有关拥有大量用户的社区的查询，同时具有超出当前图形数据管理技术支持的实时查询响应。该项目调查了索引、查询处理和存储优化技术，以在系统级别扩大此类查询的规模。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project investigates novel indexing and querying techniques that enable scientists to analyze and extract meaningful data from large repositories of user-generated data. The need for such techniques is significant, especially for user-generated social media data, which is the major repository of user-generated data and the largest archived and real-time source of human behavior and information. Thus, scientists are widely using this data in disciplines as disparate as sociology, behavioral sciences, education, spatial sciences, food sciences, medical studies, and political sciences. This project focuses on innovative indexing and querying techniques to enable scientists to effectively exploit user-generated data at a large scale.The planned research adds new data management infrastructure modules to support: (1) Scalable noise filtering queries, a subset of selection queries that are needed repeatedly and are expressed in SQL-based systems as multiple nested queries, which is not efficient for large datasets. To support this, the project investigates techniques for: (a) Advanced query conjunctions, e.g., BUT-NOT and EITHER-XOR, to scale up complex-predicate queries beyond basic search queries that are currently supported in data management systems. (b) Scalable contextual scoring of data records, e.g., based on sentiment or semantics, so irrelevant records are pruned early and the search space is downsized significantly. (2) Scalable community-centric queries that enables scientists to ask queries about communities with large numbers of users while having real-time query response beyond what is currently supported by graph data management technology. The project investigates indexing, query processing, and storage optimization techniques that scale up such queries at a system-level.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.