Collaborative Research: CCRI: New: A Research News Recommender Infrastructure with Live Users for Algorithm and Interface Experimentation

机器学习推荐系统通过根据用户过去的行为对项目进行排名和选择来个性化用户的在线体验。例如，当用户访问在线零售商时，显示的产品是由推荐系统选择的，旨在帮助用户找到要买的东西并增加供应商的销售额。推荐系统也在大多数在线新闻来源的背后，它们可以塑造人们看到的新闻。鉴于推荐系统对个人选择的重要性，研究人员能够开展研究来评估不同的设计及其对系统用户的影响是至关重要的。但是进行这样的研究超出了大多数研究人员的资源范围。为了获得有意义的结果，需要建立和维持一个自愿的用户社区，这些用户已经同意进行研究。结果，实验研究的数量——特别是对一个系统的长期用户的实验研究——直线下降。几乎所有的此类研究都是由商业推荐平台进行的，其结果很少向公众公布。该项目旨在开发一个共享新闻推荐系统，使全国的研究人员能够进行实验，并了解不同的算法和界面如何影响用户。这将创造知识，使社区能够充分理解这些系统的影响，并设计新的推荐系统，以增强公平和公平。完成后，该研究基础设施将支持研究人员回答有关复杂且通常不透明的推荐系统如何影响用户行为的关键问题，并测试可以改进这些系统及其结果的新系统。这个以社区为中心的项目将设计和构建一个实验性新闻推荐社区基础设施，以支持个性化和推荐系统、人工智能和机器学习、自然语言处理、人机交互、社会计算等领域的研究，这些领域将受益于与系统的长期用户进行在线现场实验的能力。基于云计算的软件基础设施包括一个可插拔的推荐架构，研究人员可以在其中部署自定义算法和接口，从与美联社合作获得的新闻文章开始，提供实验支持模块，包括同意，付款和调查对象，并支持两个新闻界面-首先是新闻摘要，然后是渐进式网络新闻浏览器。该基础设施将维护一组长期同意的用户，为研究人员提供广泛的支持，包括总体IRB协议、培训、样本实验、数据集和指标，并通过研究人员支持团队提供实时支持。它将由一个社区顾问委员会管理，该委员会来自研究人员社区，代表内容提供者和最终用户，负责分配实验时段，指导基础设施的开发和管理。通过开发和部署这一研究基础设施，研究人员试图授权个人和小组研究推荐系统中的重要问题，包括不同的算法和接口如何改变所代表的来源和观点的多样性，并为用户提供更好的理解和控制他们所探索的内容的问题。调查人员来自全国各地的五家机构，他们将依次组建和培训一个多元化的团队来承担这项技术上具有挑战性和重要的工作。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning recommender systems personalize users’ experiences online by ranking and selecting items to present based on users’ past behavior. For example, when a user visits an online retailer, the products shown are selected by a recommender system designed to help one find things to buy and to increase the vendors sales. Recommender systems are also behind most online news sources, and they can shape which news people see. Given the importance of recommender systems to individual choice, it is critical for researchers to be able to carry out studies to evaluate different designs and their impact on the users of the system. But conducting such studies is beyond the resources of most researchers. To get meaningful results requires building and sustaining a community of willing users who have given their permission to be studied. As a result, the amount of experimental research – and specifically experimental research on long-term users of a system – has plummeted. Almost all such studies are conducted by commercial recommendation platforms and their results are rarely made known to the public. This project is designed to develop a shared news recommender system specifically to enable researchers nationwide to be able to carry out experiments and learn just how different algorithms and interfaces affect users. This should create the knowledge that will allow the community to fully understand the impact of these systems and design new recommender systems that can enhance fairness and equity. When complete, this research infrastructure will support researchers in answering critical questions about how complex and often opaque recommender systems affect user behavior and to test new systems that can improve these systems and their outcomes.This community-centered project will design and build an experimental news recommender community infrastructure to support research in personalization and recommender systems, AI and machine learning, natural language processing, human-computer interaction, social computing, and other fields that would benefit from the ability to carry out online field experiments with long-term users of a system. The cloud-based software infrastructure includes a pluggable recommendation architecture in which researchers can deploy custom algorithms and interfaces, a feed of news articles starting with those obtained through a partnership with the Associated Press, experiment-support modules including consent, payment, and surveying of subjects, and support for two news interfaces—first a news digest and then a progressive web news browser. The infrastructure will maintain a set of long-term consented users, provide extensive support to researchers including overarching IRB protocols, training, sample experiments, datasets and metrics, and live support through a researcher support team. It will be governed by a community advisory board drawn from the researcher community with representatives of the content providers and end-users and charged with allocating experiment slots and steering the development and management of the infrastructure. By developing and deploying this research infrastructure, the investigators seek to empower individuals and small groups to study important questions in recommender systems, including questions about how different algorithms and interfaces can alter the diversity of sources and viewpoints represented and provide users with greater understanding and control over the content they explore. The investigators come from five institutions spread across the country and will in turn assemble and train a diverse team to take on this technically challenging and important work.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.