CSR--EHS: Robust Testing by Testing Robustness of Embedded Systems

CSR--EHS：通过测试嵌入式系统的鲁棒性进行鲁棒测试

• 批准号: 0720518
• 负责人: George Pappas
• 金额: $ 50万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-09-01 至 2012-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0720518&HistoricalAwards=false
• 关键词: CSR; EHS; Robust; Testing; Robustness

As modern embedded systems gain more functionality and complexity, there is a need for a novel discipline for their design, development and deployment. In recent years, the idea of the model-based design paradigm is to develop design models and subject them to early analysis, testing, and validation prior to their implementation. Simulation-based testing ensures that a finite number of user-defined system trajectories meet the desired specification. Even though computationally inexpensive simulation is ubiquitous in system design, it suffers from incompleteness, as it is impossible or impractical to test all system trajectories. On the other hand, verification methods enjoy completeness by showing that all system trajectories satisfy the desired property. For embedded hybrid systems with an infinite number of possible behaviors, exhaustive verification seems to be very hard, and simulation-based testing seems to provide no confidence in our system design. In addition to the gap between testing and verification for embedded systems, there is even a more fundamental, and largely unaddressed, challenge. Uncertainty in the environment, errors in physical devices make overall system robustness one of most critical yet least understood challenges in embedded systems. There is a clear intellectual opportunity for laying the scientific foundations and developing methods and algorithms for analyzing and testing the robustness and safety of embedded hybrid systems. This project brings together leading experts in embedded control, hybrid systems, and software monitoring and testing to develop the foundations of a modern framework for testing the robustness of embedded hybrid systems. The central idea that this proposal is centered around is the notion of a robust test, where the robustness of nominal test can be computed and used to infer that a tube of trajectories around the nominal test will yield the same qualitative behavior. By computing the robustness margins of tests, this project explores how to infer how robust each test is, guide subsequent tests, estimate the robustness for the system, as well provide well-defined coverage metrics using finite number of tests. In addition, this project emphasizes cross-cutting, multi-departmental education of graduate students and emphasizes testing and robustness for embedded hybrid systems in relevant electrical engineering and computer science courses. The educational agenda is to expose computer science students to notions of robustness, and control students to software testing algorithms.

随着现代嵌入式系统获得更多的功能和复杂性，需要一种新的学科来设计、开发和部署它们。近年来，基于模型的设计范式的思想是开发设计模型，并在其实现之前对其进行早期分析、测试和验证。基于仿真的测试确保有限数量的用户定义系统轨迹满足期望的规范。尽管计算成本低廉的模拟在系统设计中无处不在，但它存在不完整性，因为测试所有系统轨迹是不可能或不切实际的。另一方面，验证方法通过显示所有系统轨迹满足期望的性质而享有完备性。对于具有无限可能行为的嵌入式混合系统，详尽的验证似乎非常困难，并且基于仿真的测试似乎无法为我们的系统设计提供信心。除了嵌入式系统的测试和验证之间的差距之外，甚至还有一个更根本的、很大程度上没有解决的挑战。环境中的不确定性，物理设备中的错误使整个系统的稳健性成为嵌入式系统中最关键但最不为人所知的挑战之一。为嵌入式混合系统的稳健性和安全性的分析和测试奠定科学基础和开发方法和算法显然是一个智力机会。该项目汇集了嵌入式控制、混合系统和软件监控和测试方面的领先专家，为测试嵌入式混合系统的稳健性开发了一个现代框架的基础。这个提议的中心思想是围绕着一个鲁棒性测试的概念，其中标称测试的鲁棒性可以被计算出来，并用于推断在标称测试周围的一管轨迹将产生相同的定性行为。通过计算测试的健壮性边际，该项目探索了如何推断每个测试的健壮性，指导后续测试，估计系统的健壮性，以及使用有限数量的测试提供定义良好的覆盖度量。此外，本项目强调研究生的跨领域、多部门教育，并在相关的电气工程和计算机科学课程中强调嵌入式混合系统的测试和鲁棒性。教育议程是让计算机科学专业的学生了解健壮性的概念，并让学生了解软件测试算法。