Assessing the Type Annotation Burden
Assessing the Type Annotation Burden
复制标题
DOI：
10.1145/3238147.3238173
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
J. Ore;Sebastian G. Elbaum;Carrick Detweiler;Lambros Karkazis
中科院分区：
文献类型：
--
作者：
J. Ore;Sebastian G. Elbaum;Carrick Detweiler;Lambros Karkazis
类型注释提供程序变量和特定于域的类型之间的链接。当与类型系统结合使用时，这些注释可以实现早期故障检测。为了使类型注释在实践中具有成本效益，它们需要既准确又对开发人员来说负担得起。然而，我们缺乏对类型注释对开发人员来说有多么繁重的理解。因此，这项工作探讨了三个基本问题：1）开发人员如何准确地进行类型注释； 2）单个注释需要多长时间； 3）如果系统可以自动建议类型注释，正确的建议对准确性有多大好处，不正确的建议有多大危害？我们展示了 71 名程序员使用 20 个随机代码工件进行的研究结果，这些工件包含具有必须注释的物理单元类型的变量。受试者只有 51% 的时间选择正确的类型注释，并且平均需要 136 秒才能做出一次正确的注释。我们的定性分析表明，变量名称和数学运算推理是类型选择的主要线索。我们发现，建议正确的类型可将准确度提高到 73%，而提出错误的建议会使准确度降低到 28%。我们还探讨了最先进的自动化类型注释系统可以做什么和不能做什么来帮助开发人员进行类型注释，并确定对工具开发人员的影响。
Type annotations provide a link between program variables and domain-specific types. When combined with a type system, these annotations can enable early fault detection. For type annotations to be cost-effective in practice, they need to be both accurate and affordable for developers. We lack, however, an understanding of how burdensome type annotation is for developers. Hence, this work explores three fundamental questions: 1) how accurately do developers make type annotations; 2) how long does a single annotation take; and, 3) if a system could automatically suggest a type annotation, how beneficial to accuracy are correct suggestions and how detrimental are incorrect suggestions? We present results of a study of 71 programmers using 20 random code artifacts that contain variables with physical unit types that must be annotated. Subjects choose a correct type annotation only 51% of the time and take an average of 136 seconds to make a single correct annotation. Our qualitative analysis reveals that variable names and reasoning over mathematical operations are the leading clues for type selection. We find that suggesting the correct type boosts accuracy to 73%, while making a poor suggestion decreases accuracy to 28%. We also explore what state-of-the-art automated type annotation systems can and cannot do to help developers with type annotations, and identify implications for tool developers.