Approaching 100 Percent Recall for Requirements and Software Engineering Tools

需求和软件工程工具的召回率接近 100%

• 批准号: RGPIN-2016-04029
• 负责人: Berry, Daniel
• 金额: $ 2.26万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=683755
• 关键词: Approaching; 100; Percent; Recall; Requirements

A hairy requirements or software engineering task involving natural language (NL) documents is one that is not inherently difficult for NL understanding humans on a small scale but becomes unmanageable in the large scale. Examples include identification of abstractions, ambiguities, synonyms, and trace links. A hairy task demands tool assistance. Because humans need far more help in carrying out a hairy task completely than they do in making the local yes-or-no decisions, a tool for a hairy task should have as close to 100% recall (that the tool finds all desired information) as possible, even at the expense of high imprecision (that not all the information that the tool finds is desired). A tool that falls much short of 100% recall may even be useless, because to find the missing desired information, a human has to do the entire task manually anyway.***Any such tool based on NL processing (NLP) techniques inherently fails to achieve 100% recall, because even the best parsers are no more than 91% correct. Therefore, for a tool to achieve 100% recall for a hairy task, it needs to be based on something other than traditional NLP.***The reality is that a tool's achieving exactly 100% recall, which may be impossible anyway, may not be necessary. It suffices for a human working with the tool on a task to achieve better recall than a human working on the task entirely manually.***The proposed research is to discover and test a variety of non-traditional approaches to building tools for hairy tasks to see which, if any, allows a human working with the tool to achieve better recall than a human working entirely manually.***If the research succeeds, we will be able to build tools for hairy tasks that demonstrably out-perform humans on these tasks. Therefore, when a software or requirements engineer is faced with one of these hairy tasks, he or she will trust the completeness of the output of the tool and will not feel compelled to do the same task manually.**

涉及自然语言（NL）文档的复杂需求或软件工程任务对于NL在小规模上理解人类并不困难，但在大规模上变得难以管理。示例包括抽象、歧义、同义词和跟踪链接的识别。复杂的任务需要工具的帮助。因为人类在完成一项复杂任务时需要的帮助远远超过他们在做出局部是或否决定时所需要的帮助，所以用于复杂任务的工具应该尽可能具有接近100%的召回率（即该工具找到所有想要的信息），即使以高度不精确为代价（即并非所有工具找到的信息都是想要的）。一个福尔斯达不到100%召回率的工具甚至可能是无用的，因为要找到丢失的所需信息，人类无论如何都必须手动完成整个任务。任何基于NL处理（NLP）技术的工具本质上都无法实现100%的召回率，因为即使是最好的解析器也不超过91%的正确率。因此，对于一个工具来说，要实现100%的召回率，它需要基于传统NLP之外的东西。现实情况是，一个工具要达到100%的召回率，这可能是不可能的，也可能是不必要的。使用该工具完成任务的人比完全手动完成任务的人能够实现更好的回忆。*拟议的研究是发现和测试各种非传统的方法来构建用于毛茸茸的任务的工具，以查看哪些方法（如果有的话）允许使用工具的人比完全手动工作的人实现更好的回忆。如果研究成功，我们将能够为复杂的任务构建工具，这些任务的表现明显优于人类。因此，当一个软件或需求工程师面对这些棘手的任务之一时，他或她会相信工具输出的完整性，而不会感到被迫手动完成同样的任务。