EAGER: Harnessing Accurate Bias in Large-Scale Language Models

机器学习模型反映了它们所训练的数据中的模式，不幸的是，它可能会表现出负面的社会偏见，如偏见、性别歧视或种族主义。 大多数研究都试图减轻这种偏见，但这项工作颠覆了范式，并通过问：机器学习模型中的偏见可以被利用吗？有强有力的证据表明，一些语言模型表现出一种被称为“准确偏见”的特性：模型捕捉到的模式与人类的价值观、判断和观点密切相关，这些模式与时间、地理、个人身份和文化环境密切相关。事实上，相关性是如此强烈和细粒度，以至于表现出准确偏差的模型可以作为人类受试者的替代品进行研究，这意味着研究人员可以通过以人类不可能的方式对模型进行实验来获得可操作的见解。通过开发一种强大的方法和最佳实践来提取和分析语言模型中的准确偏见，有可能为社会科学开发新的工具，并可能彻底改变任何研究人类的领域，如心理学，认知科学或政治学。EARLY探索性研究基金（EAGER）将系统地研究语言模型，以确定精确偏差的可能性和局限性。作为一个EAGER，这些研究活动将是高度探索性的，旨在积累初步成果，并开发概念的技术证明，以支持未来的研究。这项工作将融合机器学习和社会科学的方法，以开发准确偏见的初步理论，以及一套配套的方法和技术最佳实践。通过研究在大规模语言模型中利用准确偏见的可行性，这项工作可以提供对人类价值观，观点和思维过程的基本见解。这项工作还可以深入了解如何改进语言模型，包括提高它们的符号推理能力，以及更深入地理解即时工程、数据策展、微调和最终模型的信息量之间的关系。 我们提案中的技术元素，如快速工程和可控文本生成，在社会科学研究背景之外具有重要的适用性，并作为机器学习社区感兴趣的进步而独立存在。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning models reflect patterns in the data they are trained on, and can, unfortunately, exhibit negative social biases such as prejudice, sexism, or racism. Most research seeks to mitigate this bias, but this work flips the paradigm and explores an alternative by asking: can the bias in machine learning models be harnessed for good? There is strong evidence that some language models exhibit a property called "accurate bias": the patterns captured by the models correlate strongly with human values, judgements, and opinions in ways that are accurately intertwined with time, geography, personal identity, and cultural milieu. In fact, the correlations are so strong and fine-grained that models exhibiting accurate bias can be studied as a surrogate for human subjects, implying researchers can derive actionable insight by experimenting on models in ways that are not possible with humans. By developing a robust methodology and best practices for extracting and analyzing the accurate bias in language models, it is possible to develop new tools for the social sciences, and could revolutionize any field that studies humans, such as psychology, cognitive science, or political science.To accomplish these goals, this EArly Grant for Exploratory Research (EAGER) will systematically study language models to determine the possibilities and limitations of accurate bias. As an EAGER, these research activities will be highly exploratory, designed to amass preliminary results and develop technical proofs of concept to support future research. The work will blend methods from machine learning and social sciences to develop a preliminary theory of accurate bias, and a suite of accompanying methodological and technical best practices. By studying the feasibility of leveraging accurate bias in large-scale language models, this work could deliver fundamental insights into the values, opinions and thought processes of humans. This work could also deliver insights into how to improve language models, including improving their ability to reason symbolically, and a deeper understanding of the relationship between prompt engineering, data curation, fine-tuning, and the informativity of the final model. Technical elements of our proposal, such as work on prompt engineering and controllable text generation, could have significant applicability outside the context of social science research, and stand on their own right as advances of interest to the machine learning community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.