AI-DCL: Collaborative Research: EAGER: Understanding and Alleviating Potential Biases in Large Scale Employee Selection Systems: The Case of Automated Video Interviews

AI-DCL：协作研究：EAGER：理解和减轻大规模员工选拔系统中的潜在偏见：自动视频面试的案例

• 批准号: 1921087
• 负责人: Sidney D'Mello
• 金额: $ 14.5万
• 依托单位: University of Colorado at Boulder
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-15 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1921087&HistoricalAwards=false
• 关键词: AI; DCL; Collaborative; Research; EAGER

The goal of this project is to use machine learning to understand and mitigate bias in interviewer evaluations. The researchers will do so by examining gender differences in expressed behavior during interviews; they will focus on behaviors that can lead to different interviewer evaluations. More specifically, they will use unsupervised video interviews to assess gender differences in terms of signaling behavior, such as facial expressions and language style; they will do so by studying how these differences are perceived by human interviewers in their indexing of personality and cognitive ability. For their research design, they rely on a large sample of men and women interviewees matched on standardized test scores of Graduate and Managerial Assessment (GMA), self-reported personality ratings, age, race, and ethnicity. The research will provide new opportunities for interdisciplinary training of students with an emphasis on recruiting underrepresented groups to work on this project. This research will provide information and guidance for developing bias-free machine-learning systems for personnel selection. By identifying and accounting for behavioral differences between genders that lead to predictive bias in machine learning selection systems, the proposed research will advance our understanding of the differences in gender expression of behaviors, methods for dealing with bias in machine learning, and bias reduction strategies in personnel selection and assessment.This project focuses on two scenarios of assessing interviewee attributes to train machine-learning algorithms: Algorithms trained on interviewee information (GMA test scores and self-reported personality), and algorithms trained on observer (interviewer) assessment of attributes. The matched sample ensures machine-learning model differences are not based on difference in underlying sample attributes. The two main goals of the project are: To understand gender differences in expressed behaviors and interviewer ratings (trained and untrained interviewers) using machine-learning techniques, and then to use that understanding to reduce predictive discrepancies between men and women by accounting for it in the models. The findings will have several significant societal impacts. They will improve our ability to predict and mitigate biases, bring to light new methodologies for mitigating bias in machine learning; and (provide strategies and tools for reducing social inequalities in employment outcomes. This research also has potential to advance both social science and machine learning. It will provide insights that can advance our understanding of social role theory by uncovering objective differences in behavior exhibited by men and women and how these behaviors are interpreted differently. Further, it can advance machine learning in developing new techniques for addressing bias at all stages of the machine-learning pipeline from instance selection and weighting, to model fitting, and then to model selection and optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的目标是使用机器学习来理解和减轻面试官评估中的偏见。研究人员将通过检查面试中表达行为的性别差异来做到这一点;他们将关注可能导致不同面试官评价的行为。更具体地说，他们将使用无监督的视频访谈来评估信号行为方面的性别差异，如面部表情和语言风格;他们将通过研究人类访谈者在人格和认知能力的索引中如何感知这些差异来做到这一点。在他们的研究设计中，他们依赖于大量的男性和女性受访者样本，这些受访者在研究生和管理评估（GMA）的标准化考试成绩、自我报告的个性评级、年龄、种族和民族上相匹配。这项研究将为学生的跨学科培训提供新的机会，重点是招募代表性不足的群体参与这一项目。这项研究将为开发无偏见的机器学习系统提供信息和指导。通过识别和解释导致机器学习选择系统中预测偏差的性别之间的行为差异，拟议的研究将促进我们对行为的性别表达差异的理解，处理机器学习中偏见的方法，以及人员选择和评估中的偏见减少策略。该项目侧重于评估受访者属性以训练机器学习算法的两种场景：根据受访者信息（GMA测试分数和自我报告的个性）训练的算法，以及根据观察者（采访者）评估属性训练的算法。匹配的样本确保机器学习模型的差异不是基于底层样本属性的差异。该项目的两个主要目标是：使用机器学习技术了解表达行为和面试官评分（受过训练和未经训练的面试官）的性别差异，然后使用这种理解来减少男性和女性之间的预测差异。这些发现将产生重大的社会影响。它们将提高我们预测和减轻偏见的能力，揭示减轻机器学习偏见的新方法;并为减少就业结果中的社会不平等提供策略和工具。这项研究还具有推动社会科学和机器学习的潜力。它将提供见解，可以通过揭示男性和女性表现出的行为的客观差异以及如何不同地解释这些行为来促进我们对社会角色理论的理解。此外，该奖项还可以促进机器学习在开发新技术方面的发展，以解决机器学习流程中从实例选择和加权到模型拟合，再到模型选择和优化的所有阶段的偏差问题。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Multimodal, Multiparty Modeling of Collaborative Problem Solving Performance

• DOI: 10.1145/3382507.3418877
• 发表时间: 2020-10
• 期刊: Proceedings of the 2020 International Conference on Multimodal Interaction
• 作者: Shree Krishna Subburaj;Angela E. B. Stewart;A. Rao;S. D’Mello

A Conceptual Framework for Investigating and Mitigating Machine-Learning Measurement Bias (MLMB) in Psychological Assessment

用于调查和减轻心理评估中机器学习测量偏差 (MLMB) 的概念框架

• DOI: 10.1177/25152459211061337
• 发表时间: 2022
• 期刊: Advances in Methods and Practices in Psychological Science
• 影响因子: 13.6
• 作者: Tay, Louis;Woo, Sang Eun;Hickman, Louis;Booth, Brandon M.;D’Mello, Sidney
• 通讯作者: D’Mello, Sidney

Toward Argument‐Based Fairness with an Application to AI‐Enhanced Educational Assessments

• DOI: 10.1111/jedm.12334
• 发表时间: 2022-06
• 期刊: Journal of Educational Measurement
• 影响因子: 1.3
• 作者: A. Huggins-Manley;Brandon M. Booth;S. D’Mello

Integrating Psychometrics and Computing Perspectives on Bias and Fairness in Affective Computing: A case study of automated video interviews

整合心理测量学和计算视角来看待情感计算中的偏见和公平：自动视频访谈的案例研究

• DOI: 10.1109/msp.2021.3106615
• 发表时间: 2021
• 期刊: IEEE Signal Processing Magazine
• 影响因子: 14.9
• 作者: Booth, Brandon M.;Hickman, Louis;Subburaj, Shree Krishna;Tay, Louis;Woo, Sang Eun;D'Mello, Sidney K.
• 通讯作者: D'Mello, Sidney K.

Psychological Measurement in the Information Age: Machine-Learned Computational Models

• DOI: 10.1177/09637214211056906
• 发表时间: 2022-02-01
• 期刊: CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE
• 影响因子: 7.2
• 作者: D'Mello, Sidney K.;Tay, Louis;Southwell, Rosy
• 通讯作者: Southwell, Rosy