Approaching 100 Percent Recall for Requirements and Software Engineering Tools

涉及自然语言(NL)文档的毛茸茸的需求或软件工程任务对于自然语言(NL)人类在小范围内理解并不是固有的困难，但在大范围内变得无法管理。例如，识别抽象、歧义、同义词和跟踪链接。一项艰巨的任务需要工具的帮助。由于人类在执行复杂任务时需要的帮助远远多于做出局部是或否决定的帮助，所以用于复杂任务的工具应该尽可能接近100%的召回率(即该工具找到所有所需的信息)，即使以高不精确度为代价(即该工具找到的并非所有信息都是所需的)。一个远远达不到100%召回率的工具甚至可能毫无用处，因为要找到丢失的所需信息，人类无论如何都必须手动完成整个任务。*任何基于自然语言处理(NLP)技术的此类工具本质上都无法实现100%召回率，因为即使是最好的语法分析器也不会超过91%的正确率。因此，对于一个复杂的任务，要实现100%的召回率，它需要基于传统NLP之外的其他东西。*现实是，一个工具实现100%的召回率可能是不必要的，这可能无论如何都是不可能的。在一项任务中，人类使用工具比完全手工操作的人能够获得更好的回忆就足够了。*这项拟议的研究是为了发现和测试各种非传统的方法来构建复杂任务的工具，看看如果有的话，哪些方法可以让使用工具的人比完全手动工作的人获得更好的回忆。*如果研究成功，我们将能够为复杂任务构建工具，显然在这些任务上表现优于人类。因此，当软件或需求工程师面临这些繁重的任务之一时，他或她将信任工具输出的完整性，而不会感到被迫手动完成相同的任务。**

A hairy requirements or software engineering task involving natural language (NL) documents is one that is not inherently difficult for NL understanding humans on a small scale but becomes unmanageable in the large scale. Examples include identification of abstractions, ambiguities, synonyms, and trace links. A hairy task demands tool assistance. Because humans need far more help in carrying out a hairy task completely than they do in making the local yes-or-no decisions, a tool for a hairy task should have as close to 100% recall (that the tool finds all desired information) as possible, even at the expense of high imprecision (that not all the information that the tool finds is desired). A tool that falls much short of 100% recall may even be useless, because to find the missing desired information, a human has to do the entire task manually anyway.***Any such tool based on NL processing (NLP) techniques inherently fails to achieve 100% recall, because even the best parsers are no more than 91% correct. Therefore, for a tool to achieve 100% recall for a hairy task, it needs to be based on something other than traditional NLP.***The reality is that a tool's achieving exactly 100% recall, which may be impossible anyway, may not be necessary. It suffices for a human working with the tool on a task to achieve better recall than a human working on the task entirely manually.***The proposed research is to discover and test a variety of non-traditional approaches to building tools for hairy tasks to see which, if any, allows a human working with the tool to achieve better recall than a human working entirely manually.***If the research succeeds, we will be able to build tools for hairy tasks that demonstrably out-perform humans on these tasks. Therefore, when a software or requirements engineer is faced with one of these hairy tasks, he or she will trust the completeness of the output of the tool and will not feel compelled to do the same task manually.**