Web Scale Trustworthy Data for Machine Learning

点击翻译按钮获取中文摘要

Large Language Models are today mostly English-language-based and trained on large-scale datasets, for which crawled web text data is the biggest element. Mojeek is an international web search engine, with an independent technology stack, and a growing search index of over 6.5 billion webpages. The Mojeek index is updated daily and data is available through the API. Our API has been used by FAIR and others to develop AI solutions including chatbots. In this project we will build a consortium of researchers and companies, for a large Phase 2 project which will enable easy and rapid access to large-scale data for training of machine learning models. We will conduct R&D, develop the Mojeek API, and work on industry use cases in chatbots and online safety. We will develop methodologies and software that will enable the extraction of trusted sources of data, thus accelerating the development of responsible AI solutions, and notably by SMEs.