KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts
KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts
复制标题
DOI：
10.1109/tvcg.2023.3346713
复制
发表时间：
2023-12
影响因子：
5.2
通讯作者：
Adam Joseph Coscia;A. Endert
中科院分区：
文献类型：
--
作者：
Adam Joseph Coscia;A. Endert
近年来，大型语言模型越来越受欢迎，导致它们在总结、预测和生成文本方面的使用越来越多，这使得帮助研究人员和工程师了解它们的工作方式和原因变得至关重要。我们提出了KnowledgeVIS，一个人在回路中的视觉分析系统，用于使用填空句作为提示来解释语言模型。通过比较句子之间的预测，KnowledgeVIS揭示了学习的关联，直观地将语言模型在训练过程中学习的内容与下游的自然语言任务联系起来，帮助用户创建和测试多个提示变体，使用新颖的语义聚类技术分析预测的单词，并使用交互式可视化发现见解。总的来说，这些可视化帮助用户识别各个预测的可能性和唯一性，比较提示之间的预测集，并总结所有提示中预测之间的模式和关系。我们展示了KnowledgeVIS的能力，来自六位NLP专家的反馈以及三个不同的用例：（1）在两个领域适应模型中探索生物医学知识;（2）评估有害的身份刻板印象;（3）发现三个通用模型之间的事实和关系。
Recent growth in the popularity of large language models has led to their increased usage for summarizing, predicting, and generating text, making it vital to help researchers and engineers understand how and why they work. We present KnowledgeVIS, a human-in-the-loop visual analytics system for interpreting language models using fill-in-the-blank sentences as prompts. By comparing predictions between sentences, KnowledgeVIS reveals learned associations that intuitively connect what language models learn during training to natural language tasks downstream, helping users create and test multiple prompt variations, analyze predicted words using a novel semantic clustering technique, and discover insights using interactive visualizations. Collectively, these visualizations help users identify the likelihood and uniqueness of individual predictions, compare sets of predictions between prompts, and summarize patterns and relationships between predictions across all prompts. We demonstrate the capabilities of KnowledgeVIS with feedback from six NLP experts as well as three different use cases: (1) probing biomedical knowledge in two domain-adapted models; and (2) evaluating harmful identity stereotypes and (3) discovering facts and relationships between three general-purpose models.