EAGER: SaTC: Early-Stage Interdisciplinary Collaboration: Fair and Accurate Information Quality Assessment Algorithm

网络空间中普遍存在的劣质信息对公民社会构成了威胁。为了提高信息质量，已经提出了多种用于在线信息质量评估的自动化算法。然而，这些算法在政治和政策观点上的公平性和表现受到了挑战，破坏了人们对这些系统的信任。通过一个独特的早期跨学科合作，将来自信息科学、计算机科学、传播、政治科学和新闻学领域的专家聚集在一起，该项目将开发准确和公平的信息质量评估算法，同时还将更深入地了解跨意识形态范围使用的信息的性质。拟议的研究推进了信息科学，并将为旨在进行自动化信息质量评估的组织提供见解，最终允许创建更安全和值得信赖的网络空间。拟议的项目将包括：(1)创建一个被牢固标记为质量和政治意识形态一致性的大型文章数据集，(2)审计现有的多种信息质量评估算法，以评估其准确性和公正性，(3)对这些算法错误标记的内容进行系统的事后归纳分析，以及(4)修改现有算法，以支持更公平和更准确的信息质量评估。这四个阶段将相互促进，利用每个学科的贡献，并将为与SATC相关的研究提供一个新的跨学科模式。该项目将为研究生提供跨学科培训，以不同的方法指导他们，并为长期的跨学科研究奠定基础。该项目还将有助于扩大对数据科学专业人员的参与。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Prevalence of poor-quality information in cyberspaces poses threats to civic society. To increase information quality, multiple automated algorithms for undertaking quality assessment of online information have been proposed. However, the fairness and performance of these algorithms across political and policy opinions has been challenged, undermining trust in such systems. Through a unique early-stage interdisciplinary collaboration that brings together experts from the fields of information science, computer science, communication, political science, and journalism, this project will develop accurate and fair information quality assessment algorithms, while also gleaning deeper insight into the nature of information being utilized across the ideological spectrum. The proposed research advances the science of information and will offer insights to organizations that aim to undertake automated information quality assessment, ultimately allowing for the creation of safer and trustworthy cyberspaces. The proposed project will include: (1) the creation of a large article dataset that has been robustly labeled for both quality and political ideological alignment, (2) an audit of multiple existing information quality assessment algorithms to assess their accuracy and fairness, (3) a systematic post-hoc inductive analysis of the content mislabeled by these algorithms, and (4) modification of existing algorithms to support fairer and more accurate information quality assessment. These four phases will build upon each other, leveraging the contributions of each discipline and will provide a new interdisciplinary model for SaTC-related research. This project will provide interdisciplinary training to graduate students, mentoring them in diverse methods and laying the groundwork for long-term interdisciplinary research. This project also will help broaden participation in data science professions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.