CIF: Medium: Foundations of Learning from Paired Comparisons and Direct Queries

CIF：媒介：配对比较和直接查询学习的基础

• 批准号: 1763734
• 负责人: Nihar Shah
• 金额: $ 119.91万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-06-01 至 2023-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1763734&HistoricalAwards=false
• 关键词: CIF; Medium; Foundations; Learning; Paired

Direct queries and paired comparisons are popular means of data acquisition in various scientific disciplines. These two types of data have been studied separately, however, several modern applications -- particularly those involving human judgments -- require analysis of a combination of these two types of data. Such an analysis presents novel opportunities and challenges for stochastic modeling, experimental design and algorithm development. This research involves establishing a unified view of learning from direct measurements and paired comparisons with the aim of understanding fundamental limits and tradeoffs for various problems of interest, and developing and implementing practical algorithms that can leverage both types of data. The utility of the algorithms will be demonstrated by application to diverse domains including material science and crowdsourcing.This project focuses on the specific technical problems of function estimation, feature selection, and optimization that can leverage passively and actively acquired direct queries and paired comparisons, with the following objectives. The first objective is to establish fundamental information-theoretic limits of learning from a combination of paired and direct measurements under various statistical models. These fundamental limits also involve understanding the inherent tradeoffs between the two forms of measurements that accounts for the different noise and costs. The second objective is to develop scalable and computationally efficient algorithms whose performance attains these fundamental limits. The third objective is to transfer the theory to practice in applications such as material design using direct experimental and pairwise expert feedback.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

直接查询和配对比较是各种科学学科中常用的数据获取方法。这两种类型的数据已经被分开研究，然而，一些现代应用——特别是那些涉及人类判断的应用——需要对这两种数据的组合进行分析。这样的分析为随机建模、实验设计和算法开发提供了新的机遇和挑战。这项研究包括建立一个统一的从直接测量和配对比较中学习的观点，目的是理解各种感兴趣的问题的基本限制和权衡，并开发和实施可以利用这两种类型的数据的实用算法。这些算法的实用性将通过在材料科学和众包等不同领域的应用来证明。该项目侧重于功能估计、特征选择和优化的具体技术问题，这些问题可以利用被动和主动获得的直接查询和配对比较，目标如下。第一个目标是在各种统计模型下，从配对和直接测量的组合中建立基本的信息论限制。这些基本限制还包括理解两种测量形式之间的内在权衡，这两种测量形式解释了不同的噪音和成本。第二个目标是开发可扩展和计算效率高的算法，其性能达到这些基本限制。第三个目标是将理论应用于实践，例如使用直接实验和成对专家反馈的材料设计。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Understanding Simultaneous Train and Test Robustness

• 发表时间: 2022
• 作者: Pranjal Awasthi;Sivaraman Balakrishnan;Aravindan Vijayaraghavan

Loss Functions, Axioms, and Peer Review

• DOI: 10.1613/jair.1.12554
• 发表时间: 2018-08
• 期刊: J. Artif. Intell. Res.
• 作者: Ritesh Noothigattu;Nihar B. Shah;Ariel D. Procaccia

Interactive martingale tests for the global null

全局零值的交互式鞅测试

• DOI: 10.1214/20-ejs1790
• 发表时间: 2020
• 期刊: Electronic Journal of Statistics
• 影响因子: 1.1
• 作者: Duan, Boyan;Ramdas, Aaditya;Balakrishnan, Sivaraman;Wasserman, Larry
• 通讯作者: Wasserman, Larry

Integrating Rankings into Quantized Scores in Peer Review

• DOI: 10.48550/arxiv.2204.03505
• 发表时间: 2022-04
• 期刊: Trans. Mach. Learn. Res.
• 作者: Yusha Liu;Yichong Xu;Nihar B. Shah;Aarti Singh

Semiparametric Counterfactual Density Estimation

• DOI: 10.1093/biomet/asad017
• 发表时间: 2021-02
• 期刊: Biometrika
• 影响因子: 2.7
• 作者: Edward H. Kennedy;Sivaraman Balakrishnan;L. Wasserman