SoD-HCER: Evaluation of Complex Designs--A Comparative Study

SoD-HCER：复杂设计的评估——比较研究

• 批准号: 0613601
• 负责人: Michael Hicks
• 金额: --
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-08-01 至 2008-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0613601&HistoricalAwards=false
• 关键词: SoD; HCER; Evaluation; Complex; Designs

A critical requirement for any science is the ability to measure. Without measurement, it is difficult to evaluate and improve. Not only is our ability to measure and evaluate software designs today weak, we also lack a thorough understanding of the shortcomings of the strengths and weaknesses of existing software design evaluation methods. To learn more about the shortcomings of current evaluation methods, the PI will in this project study two software products from the systems domain which share the important design criteria of modifiability and performance, the Click modular router and the Jetty http server and servlet container. For each of these two programs, there exist clear design goals and extensive change histories. Using change history data, the PI will create an approximate "master list" of modifiability and performance flaws in early versions of each program. For example, a feature request that required changes to many files could be flagged as a modifiability flaw. She will then apply several evaluation methods (e.g., quantitative software metrics, architecture-based evaluation, and semiautomatic methods such as design snippets) to early versions of each subject program, and compare the output of each method to the approximate master list of design flaws. Project outcomes will include establishment of a benchmark and experimental protocol that can be used to critique new techniques or tools for evaluating software designs using change history logs, and an increased understanding of the deficiencies of existing techniques and tools for evaluating software designs.Broader Impacts: Software designs matter because software matters. To improve software designs, we must have adequate tools for evaluating candidate designs. This project will provide insight into new tools and techniques for software design evaluation. Programmers using these new tools and techniques will produce better software designs and thus better software. In addition, the benchmark produced by this experiment will prove valuable to software engineering educators; instructors will be able to design assignments and projects that involve the benchmark, and which will expose students to issues in evaluating software designs and empirical software engineering research.

任何科学的一个关键要求是测量的能力。 没有测量，就很难评估和改进。 今天，我们不仅度量和评估软件设计的能力很弱，而且对现有软件设计评估方法的优缺点缺乏深入的了解。 为了更多地了解当前评估方法的缺点，PI将在本项目中研究来自系统领域的两个软件产品，它们共享可修改性和性能的重要设计标准，即Click模块化路由器和Jetty http服务器和servlet容器。 对于这两个项目中的每一个，都有明确的设计目标和广泛的变更历史。 使用变更历史数据，PI将创建每个程序早期版本的可修改性和性能缺陷的近似“主列表”。 例如，一个需要更改许多文件的功能请求可能会被标记为可修改性缺陷。 然后，她将应用几种评估方法（例如，定量软件指标、基于架构的评估和半自动方法（例如设计片段）与每个主题程序的早期版本进行比较，并将每个方法的输出与设计缺陷的大致主列表进行比较。 项目成果将包括建立一个基准和实验协议，可用于批评新技术或工具，用于评估软件设计，使用更改历史日志，并提高对现有技术和工具的缺陷的理解，用于评估软件设计。 为了改进软件设计，我们必须有足够的工具来评估候选设计。 这个项目将提供深入了解软件设计评估的新工具和技术。 使用这些新工具和技术的程序员将产生更好的软件设计，从而产生更好的软件。 此外，这个实验产生的基准将被证明是有价值的软件工程教育工作者，教师将能够设计的任务和项目，涉及基准，并将暴露学生的问题，在评估软件设计和实证软件工程研究。