SaTC: CORE: Small: Collaborative: Understanding and Mitigating Adversarial Manipulation of Content Curation Algorithms

SaTC：核心：小型：协作：理解和减轻内容管理算法的对抗性操纵

• 批准号: 1814816
• 负责人: Damon McCoy
• 金额: $ 24.16万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-15 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1814816&HistoricalAwards=false
• 关键词: SaTC; CORE; Small; Collaborative; Understanding

Online social networks (OSNs) have fundamentally transformed how billions of people use the Internet. These users are increasingly discovering books, music bands, TV shows, movies, news articles, products, and other content through posts from trusted users that they follow. All major OSNs have deployed content curation algorithms that are designed to increase interaction and act as the "gatekeepers" of what users see. While this curation and filtering is useful and necessary given the amount of content available, it has also exposed people and platforms to manipulation attacks whereby bad actors attempt to promote content people would otherwise prefer not to see. This has driven the creation of an underground ecosystem that provides services and techniques tailored towards subverting OSNs' content curation algorithms for economic and ideological gains. This project will conduct open research to improve our understanding of current algorithmic curation attackers. The team will devise content curation algorithms and defenses which are hardened against manipulation and that can be adopted by these OSN platforms, providing a systematic approach to improving design and practice in an area of critical national importance. Technology transfer from this project will protect the integrity of social media discourse from adversarial manipulation. This project will train students with expertise in security and machine learning, areas of broad national need, and produce educational materials to engage both high school students and the public in these critical questions. The team will holistically explore the economic, social, and technical perspectives of machine learning-based content curation algorithms' weaknesses. The research comprises three main activities: 1) understand how OSNs are currently being successfully manipulated at large scales, 2) investigate the defenses OSNs have in place, and 3) design more resilient defenses. The team will build the first-ever taxonomy of manipulation services and techniques that are actively used to manipulate curation algorithms. Another thrust of the project is to create a framework for the external evaluation of deployed manipulation defenses based on the collection of both public data from the OSN's platform and external data to compare it against. The team will then develop robust and scalable algorithms to detect OSN manipulation within the collected data. Finally, the team will use the insights from the taxonomy of effective manipulation techniques and the exploration of the limitation of current defenses to design fundamentally resilient content curation algorithms. The project will explore both new curation algorithms and more effective mitigation techniques for existing algorithms. The project's findings will deepen our understanding of social network manipulation and adversarial learning and produce reliable approaches to algorithmic content curation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在线社交网络（OSN）从根本上改变了数十亿人使用互联网的方式。这些用户越来越多地通过他们关注的可信用户的帖子发现书籍、乐队、电视节目、电影、新闻文章、产品和其他内容。所有主要的OSN都部署了内容策展算法，旨在增加互动，并充当用户所见内容的“守门人”。虽然考虑到可用内容的数量，这种管理和过滤是有用和必要的，但它也使人们和平台暴露在操纵攻击中，不良行为者试图推广人们不愿看到的内容。这推动了地下生态系统的创建，该生态系统提供专门针对颠覆OSN的内容策展算法以获取经济和意识形态利益的服务和技术。该项目将进行开放式研究，以提高我们对当前算法策展攻击者的理解。该团队将设计内容策展算法和防御措施，这些算法和防御措施可以被这些OSN平台采用，从而提供一种系统的方法来改进具有关键国家重要性的领域的设计和实践。该项目的技术转让将保护社交媒体话语的完整性，使其免受敌对操纵。该项目将培养学生在安全和机器学习方面的专业知识，这些领域具有广泛的国家需求，并制作教育材料，让高中生和公众参与这些关键问题。该团队将全面探索基于机器学习的内容策展算法弱点的经济，社会和技术角度。该研究包括三个主要活动：1）了解OSN目前如何被大规模成功操纵，2）调查OSN的防御措施，3）设计更具弹性的防御措施。该团队将建立有史以来第一个操纵服务和技术的分类，这些服务和技术被积极用于操纵策展算法。该项目的另一个重点是创建一个框架，用于对部署的操纵防御进行外部评估，该框架基于从OSN平台收集的公共数据和外部数据进行比较。然后，该团队将开发强大且可扩展的算法，以检测所收集数据中的OSN操作。最后，该团队将利用有效操作技术分类的见解和对当前防御限制的探索，设计具有根本弹性的内容策展算法。该项目将探索新的策展算法和现有算法的更有效的缓解技术。该项目的发现将加深我们对社交网络操纵和对抗性学习的理解，并为算法内容策展提供可靠的方法。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

An Audit of Facebook’s Political Ad Policy Enforcement

对 Facebook 政治广告政策执行情况的审计

• 发表时间: 2022
• 期刊: USENIX Security Symposium
• 作者: Le Pochat, Victor;Edelson, Laura;Van Goethem, Tom;Joosen, Wouter;McCoy, Damon;Lauinger, Tobias
• 通讯作者: Lauinger, Tobias

A Security Analysis of the Facebook Ad Library

Facebook 广告库的安全分析

• DOI: 10.1109/sp40000.2020.00084
• 发表时间: 2020
• 期刊: 2020 IEEE Symposium on Security and Privacy (SP
• 作者: Edelson, Laura;Lauinger, Tobias;McCoy, Damon
• 通讯作者: McCoy, Damon

The Pod People: Understanding Manipulation of Social Media Popularity via Reciprocity Abuse

• DOI: 10.1145/3366423.3380256
• 发表时间: 2020-04
• 期刊: Proceedings of The Web Conference 2020
• 作者: Janith Weerasinghe;Bailey Flanigan;Aviel J. Stein;Damon McCoy;R. Greenstadt

Understanding Incentivized Mobile App Installs on Google Play Store

• DOI: 10.1145/3419394.3423662
• 发表时间: 2020-10
• 期刊: Proceedings of the ACM Internet Measurement Conference
• 作者: Shehroze Farooqi;Álvaro Feal;Tobias Lauinger;Damon McCoy;Zubair Shafiq;Narseo Vallina-Rodriguez