GReaTest: Growing Readable Software Tests

测试是任何软件开发过程的关键部分。测试也是非常昂贵的：通常估计软件测试的工作量是平均预算的50%。最近的研究表明，软件开发人员花费在测试上的时间有77%用于阅读测试。测试在生成时、更新、修复或重构时、用作API使用示例和规范时或在调试期间读取。阅读和理解测试是一项挑战，有证据表明，尽管单元测试框架和测试驱动开发很流行，但大多数软件开发人员并没有积极地进行测试。自动生成的测试往往是特别不可读的，严重抑制了自动测试生成在实践中的广泛使用。测试不足的影响可能是戏剧性的，具有巨大的经济损失，并有可能伤害依赖于安全关键应用程序中的软件的人。我们提出的解决方案来解决这个问题是通过提高测试的可读性来提高测试的有效性和效率。我们将研究哪些语法和语义方面使测试可读，这样我们就可以通过建模来测量可读性，这反过来又将使我们能够提供指导手动或自动改进软件测试可读性的技术。这是通过机器学习、众包和基于搜索的测试技术的独特组合实现的。GReaTest项目将为开发人员提供工具，帮助他们识别可读性问题，自动提高可读性，并自动生成可读性优化的测试套件。可读性的重要性和可读性改进的有用性将通过与我们的行业合作者Microsoft，Google和Barclays合作的一系列实证研究进行评估，调查测试可读性与故障查找效率，开发人员生产力和软件质量的关系。测试可读性的自动分析和优化是新颖的，传统的分析只关注容易测量的程序方面，例如代码覆盖率。提高软件测试的可读性对行业有着直接的影响，在行业中，测试是一个主要的经济和技术因素：更多的可读性测试将降低测试成本，提高效率，从而提高软件质量。可读性优化将是实际自动化测试生成的关键推动因素。一旦理解了软件测试的可读性，这就为基于人类理解和性能的其他软件人工制品的分析和改进打开了新的研究方向。

Testing is a crucial part of any software development process. Testing is also very expensive: Common estimations list the effort of software testing at 50% of the average budget. Recent studies suggest that 77% of the time that software developers spend with testing is used for reading tests. Tests are read when they are generated, when they are updated, fixed, or refactored, when they serve as API usage examples and specification, or during debugging. Reading and understanding tests can be challenging, and evidence suggests that, despite the popularity of unit testing frameworks and test-driven development, the majority of software developers do not practice testing actively. Automatically generated tests tend to be particularly unreadable, severely inhibiting the widespread use of automated test generation in practice. The effects of insufficient testing can be dramatic, with large economic damage, and the potential to harm people relying on software in safety critical applications.Our proposed solution to address this problem is to improve the effectiveness and efficiency of testing by improving the readability of tests. We will investigate which syntactic and semantic aspects make tests readable, such that we can make readability measurable by modelling it. This, in turn, will allow us to provide techniques that guide manual or automatic improvement of the readability of software tests. This is made possible by a unique combination of machine learning, crowd sourcing, and search-based testing techniques. The GReaTest project will provide tools to developers that help them to identify readability problems, to automatically improve readability, and to automatically generate readability optimised test suites. The importance of readability and the usefulness of readability improvement will be evaluated with a range of empirical studies in conjunction with our industrial collaborators Microsoft, Google, and Barclays, investigating the relation of test readability to fault finding effectiveness, developer productivity, and software quality.Automated analysis and optimisation of test readability is novel, and traditional analyses only focused on easily measurable program aspects, such as code coverage. Improving readability of software tests has a direct impact on industry, where testing is a major economic and technical factor: More readable tests will reduce the costs of testing and increase effectiveness, thus improving software quality. Readability optimisation will be a key enabler for automated test generation in practice. Once readability of software tests is understood, this opens the doors to a new research direction on analysis and improvement of other software artefacts based on human understanding and performance.