Collaborative Research: CCRI: New: A Research News Recommender Infrastructure with Live Users for Algorithm and Interface Experimentation

机器学习推荐系统通过基于用户过去的行为对项目进行排名和选择来在线个性化用户体验。例如，当用户访问在线零售商时，所显示的产品由推荐系统选择，该推荐系统旨在帮助用户找到要购买的东西并增加供应商的销售。推荐系统也是大多数在线新闻来源的基础，它们可以影响人们看到的新闻。鉴于推荐系统对个人选择的重要性，研究人员必须能够进行研究，以评估不同的设计及其对系统用户的影响。但进行这样的研究超出了大多数研究人员的资源。为了获得有意义的结果，需要建立和维持一个愿意接受研究的用户社区。 结果，实验研究的数量--特别是对系统长期用户的实验研究--急剧下降。几乎所有这些研究都是由商业推荐平台进行的，其结果很少向公众公布。该项目旨在开发一个共享新闻推荐系统，专门使全国范围内的研究人员能够进行实验并了解不同的算法和界面如何影响用户。这将创造知识，使社区能够充分了解这些系统的影响，并设计新的推荐系统，可以提高公平和公正。完成后，这个研究基础设施将支持研究人员回答关于复杂且通常不透明的推荐系统如何影响用户行为的关键问题，并测试可以改善这些系统及其结果的新系统。这个以社区为中心的项目将设计和构建一个实验性新闻推荐社区基础设施，以支持个性化和推荐系统，人工智能和机器学习，自然语言处理，人机交互、社会计算和其他领域，这些领域将受益于与系统的长期用户进行在线现场实验的能力。 基于云的软件基础设施包括一个可插入的推荐架构，研究人员可以在其中部署自定义算法和界面，新闻文章的提要，从与美联社合作获得的新闻文章开始，实验支持模块，包括同意，付款和调查主题，以及支持两个新闻界面-首先是新闻摘要，然后是渐进式网络新闻浏览器。 该基础设施将维护一组长期同意的用户，为研究人员提供广泛的支持，包括总体IRB协议，培训，样本实验，数据集和指标，以及通过研究人员支持团队提供实时支持。 它将由一个社区咨询委员会管理，该委员会来自研究人员社区，有内容提供商和最终用户的代表，负责分配实验时段，指导基础设施的开发和管理。 通过开发和部署这种研究基础设施，研究人员试图使个人和小团体能够研究推荐系统中的重要问题，包括不同的算法和界面如何改变所代表的来源和观点的多样性，并为用户提供更好的理解和控制他们探索的内容。 研究人员来自全国各地的五个机构，他们将组建和培训一支多元化的团队，以承担这项具有技术挑战性和重要性的工作。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning recommender systems personalize users’ experiences online by ranking and selecting items to present based on users’ past behavior. For example, when a user visits an online retailer, the products shown are selected by a recommender system designed to help one find things to buy and to increase the vendors sales. Recommender systems are also behind most online news sources, and they can shape which news people see. Given the importance of recommender systems to individual choice, it is critical for researchers to be able to carry out studies to evaluate different designs and their impact on the users of the system. But conducting such studies is beyond the resources of most researchers. To get meaningful results requires building and sustaining a community of willing users who have given their permission to be studied. As a result, the amount of experimental research – and specifically experimental research on long-term users of a system – has plummeted. Almost all such studies are conducted by commercial recommendation platforms and their results are rarely made known to the public. This project is designed to develop a shared news recommender system specifically to enable researchers nationwide to be able to carry out experiments and learn just how different algorithms and interfaces affect users. This should create the knowledge that will allow the community to fully understand the impact of these systems and design new recommender systems that can enhance fairness and equity. When complete, this research infrastructure will support researchers in answering critical questions about how complex and often opaque recommender systems affect user behavior and to test new systems that can improve these systems and their outcomes.This community-centered project will design and build an experimental news recommender community infrastructure to support research in personalization and recommender systems, AI and machine learning, natural language processing, human-computer interaction, social computing, and other fields that would benefit from the ability to carry out online field experiments with long-term users of a system. The cloud-based software infrastructure includes a pluggable recommendation architecture in which researchers can deploy custom algorithms and interfaces, a feed of news articles starting with those obtained through a partnership with the Associated Press, experiment-support modules including consent, payment, and surveying of subjects, and support for two news interfaces—first a news digest and then a progressive web news browser. The infrastructure will maintain a set of long-term consented users, provide extensive support to researchers including overarching IRB protocols, training, sample experiments, datasets and metrics, and live support through a researcher support team. It will be governed by a community advisory board drawn from the researcher community with representatives of the content providers and end-users and charged with allocating experiment slots and steering the development and management of the infrastructure. By developing and deploying this research infrastructure, the investigators seek to empower individuals and small groups to study important questions in recommender systems, including questions about how different algorithms and interfaces can alter the diversity of sources and viewpoints represented and provide users with greater understanding and control over the content they explore. The investigators come from five institutions spread across the country and will in turn assemble and train a diverse team to take on this technically challenging and important work.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.