SaTC: CORE: Small: Collaborative: Understanding and Mitigating Adversarial Manipulation of Content Curation Algorithms

在线社交网络（osn）从根本上改变了数十亿人使用互联网的方式。这些用户越来越多地通过他们关注的可信用户的帖子发现书籍、乐队、电视节目、电影、新闻文章、产品和其他内容。所有主要的osn都部署了内容管理算法，旨在增加交互并充当用户所看到内容的“看门人”。虽然考虑到可用内容的数量，这种管理和过滤是有用和必要的，但它也使人们和平台暴露在操纵攻击之下，不良行为者试图推广人们不愿看到的内容。这推动了一个地下生态系统的创建，该生态系统提供量身定制的服务和技术，旨在颠覆osn的内容管理算法，以获得经济和意识形态收益。该项目将进行开放研究，以提高我们对当前算法策展攻击者的理解。该团队将设计内容管理算法和防御措施，这些算法和防御措施可以被这些OSN平台采用，为改善国家重要领域的设计和实践提供系统方法。该项目的技术转让将保护社交媒体话语的完整性免受敌对操纵。该项目将培养学生在安全和机器学习方面的专业知识，这些领域具有广泛的国家需求，并制作教育材料，让高中生和公众参与这些关键问题。该团队将从经济、社会和技术角度全面探索基于机器学习的内容管理算法的弱点。该研究包括三个主要活动：1)了解当前如何成功地大规模操纵osn， 2)调查osn的防御措施，以及3)设计更具弹性的防御措施。该团队将建立有史以来第一个操作服务和技术的分类，这些服务和技术被积极用于操作策展算法。该项目的另一个重点是创建一个框架，用于对部署的操作防御进行外部评估，该评估基于来自OSN平台的公共数据和用于比较的外部数据的集合。然后，该团队将开发健壮且可扩展的算法来检测收集数据中的OSN操作。最后，该团队将利用从有效操作技术的分类中获得的见解，并探索当前防御的局限性，以设计具有基本弹性的内容管理算法。该项目将探索新的策展算法和现有算法的更有效的缓解技术。该项目的研究结果将加深我们对社交网络操纵和对抗性学习的理解，并为算法内容管理提供可靠的方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Online social networks (OSNs) have fundamentally transformed how billions of people use the Internet. These users are increasingly discovering books, music bands, TV shows, movies, news articles, products, and other content through posts from trusted users that they follow. All major OSNs have deployed content curation algorithms that are designed to increase interaction and act as the "gatekeepers" of what users see. While this curation and filtering is useful and necessary given the amount of content available, it has also exposed people and platforms to manipulation attacks whereby bad actors attempt to promote content people would otherwise prefer not to see. This has driven the creation of an underground ecosystem that provides services and techniques tailored towards subverting OSNs' content curation algorithms for economic and ideological gains. This project will conduct open research to improve our understanding of current algorithmic curation attackers. The team will devise content curation algorithms and defenses which are hardened against manipulation and that can be adopted by these OSN platforms, providing a systematic approach to improving design and practice in an area of critical national importance. Technology transfer from this project will protect the integrity of social media discourse from adversarial manipulation. This project will train students with expertise in security and machine learning, areas of broad national need, and produce educational materials to engage both high school students and the public in these critical questions. The team will holistically explore the economic, social, and technical perspectives of machine learning-based content curation algorithms' weaknesses. The research comprises three main activities: 1) understand how OSNs are currently being successfully manipulated at large scales, 2) investigate the defenses OSNs have in place, and 3) design more resilient defenses. The team will build the first-ever taxonomy of manipulation services and techniques that are actively used to manipulate curation algorithms. Another thrust of the project is to create a framework for the external evaluation of deployed manipulation defenses based on the collection of both public data from the OSN's platform and external data to compare it against. The team will then develop robust and scalable algorithms to detect OSN manipulation within the collected data. Finally, the team will use the insights from the taxonomy of effective manipulation techniques and the exploration of the limitation of current defenses to design fundamentally resilient content curation algorithms. The project will explore both new curation algorithms and more effective mitigation techniques for existing algorithms. The project's findings will deepen our understanding of social network manipulation and adversarial learning and produce reliable approaches to algorithmic content curation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.