A Study on Subjective Information in Deep Neural Network Models
A Study on Subjective Information in Deep Neural Network Models
复制标题
深度神经网络模型中主观信息的研究
DOI：
10.1109/candarw57323.2022.00077
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Yamaguchi Saneyasu
中科院分区：
文献类型：
--
作者：
Kobayashi Atsuya;Takahashi Yoshihaya;Yamaguchi Saneyasu
深度学习技术在过去十年中取得了显著的进步，自然语言处理（NLP）技术也是如此。预学习模型已经取得了重大进展，现在正被非常积极地使用。随着预训练模型的积极使用，模型中包含的知识得到了积极的研究，以揭示其决策的基础。然而，对这些知识的考察研究主要是针对客观知识的，对模型所包含的主观知识的考察研究还不够。本文以目前最流行的预训练模型之一GPT-3为研究对象，通过让GPT-3生成英语或日语句子来研究其模型中包含的主观知识。为了进行调查，我们检查了是否产生了对各政党有利的资料。”结果表明，该模型对每个政党都有主观认识，可以判断政党是好是坏。我们还发现，让模型生成英语文档比生成日语文档更能提取主观知识。
Deep learning technology has advanced remarkably in the last decade, and so has natural language processing (NLP) technology. Pre-learning models have achieved significant advances and are now being used very actively. As pre-trained models have been actively used, the knowledge contained in the models has been actively studied in order to reveal the basis for their decisions. However, the research on the investigation of such knowledge is mainly for objective knowledge, and there is not enough studies on the investigation of the subjective knowledge that the model contains. In this paper, we focus on GPT-3, which is one of the most popular pre-trained models, and investigate the subjective knowledge contained in its model by making GPT-3 generate sentences in English or Japanese. For investigation, we checked whether it generates documents favorable to each political party. The results showed that the model has subjective knowledge about each political party, which indicates whether the party is good or bad. We also found that subjective knowledge could be extracted more by making the model generate documents in English than in Japanese.