CAREER: Fundamentals of Learning from People with Applications to Peer Review
CAREER: Fundamentals of Learning from People with Applications to Peer Review
批准号:
1942124
负责人:
Nihar Shah
金额:
$64.9万
依托单位国家:
美国
项目类别:
Continuing Grant
财政年份:
2020
资助国家:
美国
项目状态:
未结题
起止时间:
2020-02-01 至 2025-01-31
中文摘要
点击翻译按钮获取中文摘要
英文摘要
In a wide variety of applications -- such as peer review, recommender systems, hiring, college admissions, peer grading, A/B testing, and crowdsourcing -- it is common to elicit and process data from people. Such data often suffer from issues such as miscalibration, subjectivity, strategic behavior, and biases. These issues are further amplified in applications (such as those listed above) in which the data comprises evaluations of a set of items by people, where every person evaluates only a subset of items and every item is evaluated by only a subset of people. These issues degrade the overall quality of these applications and also lead to unfairness towards some of its participants. For example, data from people often have biases pertaining to certain demographics; subjective opinions or strictness/leniency of the human evaluators can lead to unfairness; some participants may indulge in strategic behavior which can be detrimental to the overall system. This project will design algorithms for eliciting data from people and processing it in a manner that mitigates these issues to the maximum possible extent. The project will have a particular focus on the application of peer review of scholarly research. It will make a significant real-world impact through the research outcomes for applications that depend on data from people, outreach to drive positive policy changes, and synergistic educational activities.This project will address the issues of miscalibration, subjectivity, strategic behavior and biases in learning from people along three fronts. First, using tools from information theory and statistics, it will establish the fundamental limits on the extent to which these problems can be mitigated. Second, it will develop algorithms that will provably achieve (or approach) these limits, and are also computationally efficient. The research on this front will employ tools from machine learning and statistics, game theory and social choice theory. Finally, the project will transform the theory into a useful toolkit for practitioners, as well as outreach towards driving positive policy changes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
期刊论文(22)
专著(0)
科研奖励(0)
会议论文
登录
查看更多内容
DOI:
10.1609/aaai.v35i6.16611
发表时间:
2020-10
期刊:
ArXiv
影响因子:
--
作者:
[Ivan Stelmakh;Nihar B. Shah;Aarti Singh]
通讯作者:
Ivan Stelmakh;Nihar B. Shah;Aarti Singh
DOI:
10.48550/arxiv.2204.03505
发表时间:
2022-04
期刊:
Trans. Mach. Learn. Res.
影响因子:
--
作者:
[Yusha Liu;Yichong Xu;Nihar B. Shah;Aarti Singh]
通讯作者:
Yusha Liu;Yichong Xu;Nihar B. Shah;Aarti Singh
DOI:
10.48550/arxiv.2302.08450
发表时间:
2023-02
期刊:
ArXiv
影响因子:
--
作者:
[Joon Sik Kim;Valerie Chen;Danish Pruthi;Nihar B. Shah;Ameet Talwalkar]
通讯作者:
Joon Sik Kim;Valerie Chen;Danish Pruthi;Nihar B. Shah;Ameet Talwalkar
A Heuristic for Statistical Seriation
统计序列化的启发式
DOI:
--
发表时间:
2021
期刊:
UAI
影响因子:
--
作者:
[Dhull, Komal, Wang, Jingyan, Shah, Nihar, Li, Yuanzhi, and Ravi, R]
通讯作者:
and Ravi, R
DOI:
--
发表时间:
2018-06
期刊:
J. Mach. Learn. Res.
影响因子:
--
作者:
[Ivan Stelmakh;Nihar B. Shah;Aarti Singh]
通讯作者:
Ivan Stelmakh;Nihar B. Shah;Aarti Singh
共 18 条
RI: Small: Robustness to Undesirable Behavior in Peer Review
-
批准号:2200410
-
项目类别:Standard Grant
-
资助金额:$60.0万
-
财政年份:2022
-
负责人:Nihar Shah
-
依托单位:
CRII: CIF: Crowdsourcing-aware Learning
-
批准号:1755656
-
项目类别:Standard Grant
-
资助金额:$17.49万
-
财政年份:2018
-
负责人:Nihar Shah
-
依托单位:
CIF: Medium: Foundations of Learning from Paired Comparisons and Direct Queries
-
批准号:1763734
-
项目类别:Continuing Grant
-
资助金额:$119.91万
-
财政年份:2018
-
负责人:Nihar Shah
-
依托单位:
国内基金
海外基金
The Heterogenous Impact of Monetary Policy on Firms' Risk and Fundamentals
-
批准号:--
-
项目类别:外国学者研究基金项目
-
资助金额:--
-
批准年份:2024
-
负责人:潘军
-
依托单位: