III: Medium: Dataset Search and Ranking for Data Augmentation and Explanation

III：中：用于数据增强和解释的数据集搜索和排序

• 批准号: 2106888
• 负责人: Juliana Freire
• 金额: $ 109.32万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106888&HistoricalAwards=false
• 关键词: III; Medium; Dataset; Search; Ranking

There has been an explosion in the volume of data that is being collected and cataloged about the environment, society, and populace. Moreover, with the push towards transparency and open data, scientists, governments, and organizations are increasingly making these data available on the Web. Combined with advances in analytics and machine learning, such growing access to data should in theory allow for progress on many of the world’s most important scientific and societal questions. However, this opportunity is often missed due to a central technical barrier: it is currently nearly impossible for domain experts to weed through the vast amount of publicly-available information to discover datasets that are needed for their specific application. Data repository platforms, such as CKAN and Dataverse, and dataset search engines, such as Google Dataset Search, aim to make it easy to share and find datasets. But these systems only support simple, keyword-based queries and metadata search, which are insufficient for users to properly specify their information needs. The investigators envision a new kind of dataset search engine that unlocks the untapped value in open data by supporting a richer set of findability queries that cater to the needs of analytics tasks, and aid in the construction and refinement of machine learning models. By empowering scientists and practitioners with the ability to discover relevant data, the project has great potential to stimulate data reuse both within and across domains.The project will develop methods where the user’s existing data forms the basis of a query that retrieves additional, related data from a large collection of datasets and attributes. There are many technical hurdles to overcome to support such queries. One primary challenge is computational efficiency: this project will develop novel algorithms for rapidly computing and searching for dataset relationships. The investigators will build on a rich variety of tools, including randomized sketching and hashing algorithms, and contribute new theoretical analyses to understand these methods. The algorithms contributed will address both highly-structured data (e.g., spatio-temporal) as well as generic numerical or categorical data. A second challenge is usability: the project will develop novel methods for assessing the significance of discovered data relationships, for pruning out coincidental or spurious relationships, and for ranking and presenting datasets to the end-user. Finally, the project will contribute a formalism to the dataset search problem that supports a wide range of findability queries based on dataset relationships. Active plans for engagement in STEM related activities for high-school students are detailed.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

关于环境、社会和民众的数据被收集和编目的数量激增。此外，随着对透明度和开放数据的推动，科学家、政府和组织越来越多地将这些数据放在网上。结合分析和机器学习的进步，这种对数据的日益增长的访问在理论上应该允许在世界上许多最重要的科学和社会问题上取得进展。然而，由于一个核心技术障碍，这个机会经常被错过：目前领域专家几乎不可能通过大量的公开信息来发现其特定应用所需的数据集。数据存储库平台，如CKAN和Dataverse，以及数据集搜索引擎，如Google Dataset Search，旨在使共享和查找数据集变得容易。但这些系统只支持简单的、基于关键字的查询和元数据搜索，不足以让用户正确指定其信息需求。研究人员设想了一种新的数据集搜索引擎，通过支持更丰富的可查找性查询来释放开放数据中未开发的价值，以满足分析任务的需求，并帮助构建和完善机器学习模型。通过赋予科学家和从业人员发现相关数据的能力，该项目具有极大的潜力，可以促进数据在领域内和跨领域的重用。该项目将开发方法，使用户现有的数据形成查询的基础，从大量的数据集和属性中检索额外的相关数据。要支持这种查询，需要克服许多技术障碍。一个主要的挑战是计算效率：该项目将开发快速计算和搜索数据集关系的新算法。研究人员将建立在丰富的各种工具，包括随机草图和散列算法，并贡献新的理论分析，以了解这些方法。贡献的算法将解决高度结构化的数据（例如，空间-时间）以及通用数值或分类数据。第二个挑战是可用性：该项目将开发新的方法来评估发现的数据关系的重要性，修剪掉巧合或虚假的关系，并将数据集排序和呈现给最终用户。最后，该项目将有助于形式主义的数据集搜索问题，支持广泛的基于数据集关系的可查找性查询。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Simple Analysis of Priority Sampling

优先采样的简单分析

• 发表时间: 2024
• 期刊: SIAM Symposium on Simplicity in Algorithms
• 作者: Majid Daliri;Juliana Freire;Christopher Musco;Aécio Santos;Haoxiang Zhang
• 通讯作者: Haoxiang Zhang

Weighted Minwise Hashing Beats Linear Sketching for Inner Product Estimation

• DOI: 10.1145/3584372.3588679
• 发表时间: 2023-01
• 期刊: Proceedings of the 42nd ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems
• 作者: Aline Bessa;Majid Daliri;Juliana Freire;Cameron Musco;Christopher Musco;Aécio S. R. Santos;H. Zhang

A Sketch-based Index for Correlated Dataset Search

用于相关数据集搜索的基于草图的索引

• DOI: 10.1109/icde53745.2022.00264
• 发表时间: 2022
• 期刊: 2022 IEEE 38th International Conference on Data Engineering (ICDE
• 作者: Santos, Aecio;Bessa, Aline;Musco, Christopher;Freire, Juliana
• 通讯作者: Freire, Juliana