EAGER: Requirements Domain Specifications for Machine-Learned Software Components

传统上，软件组件是根据一组预定义的规范构建的，这些规范定义了在给定的上下文中期望软件组件做什么。这些规范通常是在领域分析和需求工程阶段从涉众和客户那里收集的。与确定性软件组件相比，使用机器学习（ML）算法构建的软件组件从一组收集的示例中学习其规范，而不是从一组商定的规范中学习。当支持ml的软件的功能正确性仅取决于训练数据时，现实世界概念的规范与收集的数据集表示的概念之间可能存在显著差距。本研究的目标是定义具有机器学习组件的软件的需求满足的含义，并研究设计这些需求的方法。在这个项目中，研究人员将正式指定机器学习组件（mlc）的部分需求，而不是允许他们以一种特别的方式从一组收集的样本中单独学习这些规范。因此，目标是通过用领域分析增强ML的归纳性质，使机器学习组件更好地满足需求，以表征数据集包含或缺乏满足需求所需的重要特征的程度。该项目提供了一个框架，用于正式指定部分需求，以及验证收集样本中此类规范的存在，这本质上表征了数据集包含或缺乏对学习任务重要的特征的程度。该提案在自动驾驶系统的背景下考虑了这个问题，在自动驾驶系统中，正确定义现实世界的概念对于安全来说至关重要。例如，正确的图像识别需要对要避开的物体进行分类；该奖项的目的是表明，将数据集训练与提取需求的部分领域模型相结合，可以胜过暴力机器学习。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Software components are traditionally built according to a set of pre-defined specifications that define what a software component is expected to do in a given context. These specifications are traditionally gathered from stakeholders and customers during the domain-analysis and requirements-engineering phases. In contrast to deterministic software components, software components built using Machine-Learning (ML) algorithms learn their specifications from a set of collected examples rather than a set of agreed upon specifications. When the functional correctness of ML-enabled software depends only on the training data, there can be a significant gap between specification of a real-world concept and what a collected dataset represents as the concept. The goal of this research is to define the meaning of requirements satisfaction for software with machine-learning components, and to investigate methods for engineering those requirements. In this project, the investigators will formally specify partial requirements for Machine-Learned Components (MLCs) instead of allowing them to learn these specifications solely from a set of collected samples in an ad-hoc manner. The goal is thus to make machine-learning components better meet requirements by augmenting the inductive nature of ML with domain analysis, in order to characterize the extent to which the dataset contains or lacks important features that are necessary to meet requirements. This project provides a framework for formally specifying partial requirements as well as validating the presence of such specifications in the collected samples, which in essence characterizes the extent to which the dataset contains or lacks features important to learning the task. The proposal considers this problem in the context of automated driving systems where the correct definition of real-world concepts is critically important for safety reasons. For example, correct image recognition is needed to classify objects to avoid; the objective is to show that combining training by datasets with partial domain models from elicited requirements can outperform brute-force ML.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.