Approaching 100 Percent Recall for Requirements and Software Engineering Tools

涉及自然语言(NL)文档的毛茸茸的需求或软件工程任务对于自然语言(NL)人类在小范围内理解并不是固有的困难，但在大范围内变得无法管理。例如，识别抽象、歧义、同义词和跟踪链接。一项艰巨的任务需要工具的帮助。由于人类在执行复杂任务时需要的帮助远远多于做出局部是或否决定的帮助，所以用于复杂任务的工具应该尽可能接近100%的召回率(即该工具找到所有所需的信息)，即使以高不精确度为代价(即该工具找到的并非所有信息都是所需的)。一个远达不到100%召回率的工具甚至可能毫无用处，因为要找到丢失的所需信息，人类无论如何都必须手动完成整个任务。 任何这样的基于自然语言处理(NLP)技术的工具都无法实现100%的召回，因为即使是最好的解析器也不会超过91%的正确率。因此，对于一个毛茸茸的任务实现100%召回的工具，它需要基于传统NLP之外的东西。 现实情况是，一个工具完全达到100%的召回率可能是不可能的，可能并不是必要的。与完全手动处理任务的人相比，使用该工具处理任务的人可以实现更好的召回率。 这项拟议的研究是为了发现和测试各种非传统的方法来构建毛茸茸的任务工具，看看如果有的话，哪些方法可以让使用工具的人比完全手动工作的人获得更好的回忆。 如果研究成功，我们将能够为毛茸茸的任务制造工具，在这些任务上明显超过人类。因此，当软件或需求工程师面临这些繁琐的任务之一时，他或她将信任工具输出的完整性，而不会感到被迫手动完成相同的任务。

A hairy requirements or software engineering task involving natural language (NL) documents is one that is not inherently difficult for NL understanding humans on a small scale but becomes unmanageable in the large scale. Examples include identification of abstractions, ambiguities, synonyms, and trace links. A hairy task demands tool assistance. Because humans need far more help in carrying out a hairy task completely than they do in making the local yes-or-no decisions, a tool for a hairy task should have as close to 100% recall (that the tool finds all desired information) as possible, even at the expense of high imprecision (that not all the information that the tool finds is desired). A tool that falls much short of 100% recall may even be useless, because to find the missing desired information, a human has to do the entire task manually anyway. Any such tool based on NL processing (NLP) techniques inherently fails to achieve 100% recall, because even the best parsers are no more than 91% correct. Therefore, for a tool to achieve 100% recall for a hairy task, it needs to be based on something other than traditional NLP. The reality is that a tool's achieving exactly 100% recall, which may be impossible anyway, may not be necessary. It suffices for a human working with the tool on a task to achieve better recall than a human working on the task entirely manually. The proposed research is to discover and test a variety of non-traditional approaches to building tools for hairy tasks to see which, if any, allows a human working with the tool to achieve better recall than a human working entirely manually. If the research succeeds, we will be able to build tools for hairy tasks that demonstrably out-perform humans on these tasks. Therefore, when a software or requirements engineer is faced with one of these hairy tasks, he or she will trust the completeness of the output of the tool and will not feel compelled to do the same task manually.