Toward Imitating Visual Attention of Experts in Software Development Tasks
Toward Imitating Visual Attention of Experts in Software Development Tasks
复制标题
DOI：
10.1109/emip.2019.00013
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Yoshiharu Ikutani;Nishanth Koganti;Hideaki Hata;Takatomi Kubo;Ken-ichi Matsumoto
中科院分区：
文献类型：
--
作者：
Yoshiharu Ikutani;Nishanth Koganti;Hideaki Hata;Takatomi Kubo;Ken-ichi Matsumoto
专家程序员在阅读源代码期间的眼球运动是被认为与他们的领域专长相关联的有价值的来源。我们提倡一种新的智能系统的愿景，将专家的专业知识用于软件开发任务，如问题本地化，注释生成和代码生成。我们提出了一个基于模仿学习（IL）的神经自主代理的概念框架，它使代理能够通过他/她的眼球运动来模仿专家的视觉注意。在该框架中，自主代理被构造为基于上下文的注意模型，该模型由编码器/解码器网络组成，并使用专家演示产生的状态-动作序列进行训练。本文讨论了实现一个基于IL的自主代理专门为软件开发任务的挑战。
Expert programmers' eye-movements during source code reading are valuable sources that are considered to be associated with their domain expertise. We advocate a vision of new intelligent systems incorporating expertise of experts for software development tasks, such as issue localization, comment generation, and code generation. We present a conceptual framework of neural autonomous agents based on imitation learning (IL), which enables agents to mimic the visual attention of an expert via his/her eye movement. In this framework, an autonomous agent is constructed as a context-based attention model that consists of encoder/decoder network and trained with state-action sequences generated by an experts' demonstration. Challenges to implement an IL-based autonomous agent specialized for software development task are discussed in this paper.