Detox: Human-led AI to automate and radically improve online content moderation

在这个项目中，Rewire Online Limited将开发一种新的人工智能(AI)驱动的产品_Detox_，并进行商业试验，该产品将极大地改善展示平台适度的在线内容。排毒将自动检测和评级有毒内容，在短时间内为大量内容提供准确、可靠和信息丰富的标签。这项技术有可能改变有毒在线内容的处理方式，极大地提高人们的在线安全，并减少人类版主承担的高风险工作量。它还将缓解平台面临的商业、法律、声誉、道德和成本压力，这些平台几乎都在努力处理恶意和不受欢迎的内容。大多数平台仍然严重依赖人工分析师来缓和有毒内容，因为没有供应商开发出能够自动给出准确评级的系统。大多数现有的商业解决方案都不可信，缺乏精确度和覆盖面，而且很快就会过时。我们的人工智能技术不同于目前的产品，因为它是由人主导的。它动态地将数据收集与模型训练相结合，从而实现了大幅的性能改进，同时降低了产品开发的时间和成本。使用传统的人工智能方法，分析师只需标记有毒内容，即可创建已标记的训练数据集。相比之下，我们的分析员的任务是创建敌意内容，他们认为这将‘欺骗’人工智能。他们通过创建人工智能认为有毒但实际上没有的内容来做到这一点，反之亦然。通过这种方式，注释者识别并利用人工智能的弱点。然后我们更新人工智能，反复重复这个过程。这项技术是在研究背景下开发的，现在已经准备好进行商业试验。Detox满足了对高质量和高成本效益的自动化内容审核工具的日益增长的需求，这些工具实际上是有效的。它代表了在线内容自动审核方面的真正突破，兑现了人工智能在该领域的巨大但未兑现的承诺。该项目将创造众多的社会、经济和创新效益，使英国走在确保网络安全的创新解决方案的前沿。

In this project, Rewire Online Limited will develop and commercially trial a new Artificial Intelligence (AI)-powered product, _Detox_, which massively improve show platforms moderate online content. Detox will automatically detect and rate toxic content, giving accurate, reliable and informative labels to large volumes in a short space of time. This technology has the potential to create a step-change in how toxic online content is tackled, dramatically improving people's safety online and reducing the amount of risky work undertaken by human moderators. It will also alleviate commercial, legal, reputational, ethical and cost pressures on platforms, which are nearly all struggling to deal with malicious and unwanted content.Most platforms still rely heavily on human analysts to moderate toxic content because no provider has developed a system which can automatically give accurate ratings. Most existing commercial solutions are not trusted, lacking precision, coverage and quickly going out of date. Our AI technology is different from current offerings because it is human-led. It dynamically integrates data collection with model training, which enables substantial performance improvements whilst decreasing the time and cost of product development. With a traditional approach to AI, analysts just label toxic content to create a labelled training dataset. In contrast, our analysts are tasked with creating adversarial content which they think will 'trick' the AI. They do this by creating content which the AI thinks is toxic but actually is not, and vice versa. In this way the annotators identify and exploit the AI's weaknesses. We then update the AI, iteratively repeating the process. This technology has been developed in a research context, and is now ready to be trialled commercially. Detox addresses the growing need for high-quality and cost-effective automated content moderation tools that _actually work_. It presents a genuine breakthrough in the automatic moderation of online content, delivering on the huge but unfulfilled promise of AI in this field. This project will create numerous social, economic and innovation benefits, placing the UK at the forefront of innovative solutions to ensure online safety.