Configurations in Android testing: they matter
Configurations in Android testing: they matter
复制标题
DOI：
10.1145/3243218.3243219
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
Emily Kowalczyk;Myra B. Cohen;A. Memon
中科院分区：
文献类型：
--
作者：
Emily Kowalczyk;Myra B. Cohen;A. Memon
Android在很大程度上要归功于其开源模式，它已经飞升到移动市场的顶端。供应商免费在他们的设备上使用Android，公司根据他们的需求进行定制。这导致了目前存在于用户空间中的无数配置。在本文中，我们展示了配置上的差异，如果忽略的话，会导致测试输出和代码覆盖率的差异。因此，开发新测试技术并仅在一两个配置上对其进行评估的研究人员在他们的实验中遗漏了必要的维度，而忽视这一点的开发人员可能会发布有缺陷的软件。在一项针对88种配置的18款应用的大型研究中，我们发现，在被研究的18款应用中，只有一款完全没有变化。其余的显示了代码覆盖率和测试结果的变化，或者两者都有。在所有应用的2000多个测试用例中，有15%是不同的，其中一些变化是微妙的，例如不仅仅是测试崩溃。我们的研究结果表明，Android测试中的配置确实很重要，开发者需要使用配置感知技术进行测试。
Android has rocketed to the top of the mobile market thanks in large part to its open source model. Vendors use Android for their devices for free, and companies make customizations to suit their needs. This has resulted in a myriad of configurations that are extant in the user space today. In this paper, we show that differences in configurations, if ignored, can lead to differences in test outputs and code coverage. Consequently, researchers who develop new testing techniques and evaluate them on only one or two configurations are missing a necessary dimension in their experiments and developers who ignore this may release buggy software. In a large study on 18 apps across 88 configurations, we show that only one of the 18 apps studied showed no variation at all. The rest showed variation in either, or both, code coverage and test results. 15% of the 2,000 plus test cases across all of the apps vary, and some of the variation is subtle, i.e. not just a test crash. Our results suggest that configurations in Android testing do matter and that developers need to test using configuration-aware techniques.