Detox: Human-led AI to automate and radically improve online content moderation

Detox：以人为主导的人工智能实现自动化并从根本上改善在线内容审核

• 批准号: 10003630
• 金额: $ 26.54万
• 依托单位: REWIRE ONLINE LIMITED
• 依托单位国家: 英国
• 项目类别: Feasibility Studies
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=10003630
• 关键词: Detox; Human; led; AI; automate

In this project, Rewire Online Limited will develop and commercially trial a new Artificial Intelligence (AI)-powered product, _Detox_, which massively improve show platforms moderate online content. Detox will automatically detect and rate toxic content, giving accurate, reliable and informative labels to large volumes in a short space of time. This technology has the potential to create a step-change in how toxic online content is tackled, dramatically improving people's safety online and reducing the amount of risky work undertaken by human moderators. It will also alleviate commercial, legal, reputational, ethical and cost pressures on platforms, which are nearly all struggling to deal with malicious and unwanted content.Most platforms still rely heavily on human analysts to moderate toxic content because no provider has developed a system which can automatically give accurate ratings. Most existing commercial solutions are not trusted, lacking precision, coverage and quickly going out of date. Our AI technology is different from current offerings because it is human-led. It dynamically integrates data collection with model training, which enables substantial performance improvements whilst decreasing the time and cost of product development. With a traditional approach to AI, analysts just label toxic content to create a labelled training dataset. In contrast, our analysts are tasked with creating adversarial content which they think will 'trick' the AI. They do this by creating content which the AI thinks is toxic but actually is not, and vice versa. In this way the annotators identify and exploit the AI's weaknesses. We then update the AI, iteratively repeating the process. This technology has been developed in a research context, and is now ready to be trialled commercially. Detox addresses the growing need for high-quality and cost-effective automated content moderation tools that _actually work_. It presents a genuine breakthrough in the automatic moderation of online content, delivering on the huge but unfulfilled promise of AI in this field. This project will create numerous social, economic and innovation benefits, placing the UK at the forefront of innovative solutions to ensure online safety.

在这个项目中，Rewire Online Limited将开发和商业试验一种新的人工智能（AI）驱动的产品_Detox_，这将大大改善展示平台的适度在线内容。Detox将自动检测和评估有毒物质含量，在短时间内为大量产品提供准确、可靠和信息丰富的标签。这项技术有可能在如何处理有毒在线内容方面产生一个步骤性的变化，大大提高人们的在线安全，并减少人类版主承担的风险工作量。它还将减轻平台在商业、法律的、声誉、道德和成本方面的压力，这些平台几乎都在努力处理恶意和不受欢迎的内容。大多数平台仍然严重依赖人工分析师来缓和有毒内容，因为没有一家提供商开发出可以自动给出准确评级的系统。大多数现有的商业解决方案都不值得信赖，缺乏精确性、覆盖范围并且很快就会过时。我们的人工智能技术与目前的产品不同，因为它是人类主导的。它动态地将数据收集与模型训练集成在一起，从而在降低产品开发时间和成本的同时实现了实质性的性能改进。在传统的人工智能方法中，分析师只是标记有毒内容来创建标记的训练数据集。相比之下，我们的分析师的任务是创建他们认为会“欺骗”人工智能的对抗性内容。他们通过创建人工智能认为有毒但实际上没有的内容来做到这一点，反之亦然。通过这种方式，注释者可以识别并利用人工智能的弱点。然后我们更新AI，迭代地重复这个过程。这项技术是在研究背景下开发的，现在已经准备好进行商业试验。Detox解决了对高质量和高性价比的自动内容审核工具的日益增长的需求。它在在线内容的自动审核方面取得了真正的突破，实现了人工智能在这一领域巨大但尚未实现的承诺。该项目将创造众多的社会、经济和创新效益，使英国处于确保在线安全的创新解决方案的最前沿。