论文标题
JavaScript中片状测试的经验研究
An Empirical Study of Flaky Tests in JavaScript
论文作者
论文摘要
片状测试（具有非确定性结果的测试）对于测试效率和软件可靠性可能是有问题的。测试套件中的片状测试也可以显着延迟软件发布。有几项研究试图量化不同编程语言（例如Java和Python）和应用领域（例如，基于移动和GUI）的测试片段的影响。在本文中，我们对JavaScript中的片状测试状态进行了经验研究。我们研究了JavaScript项目中片状测试的两个方面：这些项目中片状测试的主要原因和常见的修复策略。通过分析Github的大型，最高得分的JavaScript项目的452个提交，我们发现与并发有关的问题（例如，异步等待，种族条件或僵局）引起的片状是测试散布的最主要原因。片状测试的其他主要原因是操作系统特定的（例如，在特定的OS或OS版本上起作用的功能）和网络稳定性（例如，Internet可用性或不良插座管理）。关于如何处理片状测试，大多数片状测试（> 80％）是固定的，以消除片状的行为，而开发人员有时会跳过，隔离或删除片状测试。
Flaky tests (tests with non-deterministic outcomes) can be problematic for testing efficiency and software reliability. Flaky tests in test suites can also significantly delay software releases. There have been several studies that attempt to quantify the impact of test flakiness in different programming languages (e.g., Java and Python) and application domains (e.g., mobile and GUI-based). In this paper, we conduct an empirical study of the state of flaky tests in JavaScript. We investigate two aspects of flaky tests in JavaScript projects: the main causes of flaky tests in these projects and common fixing strategies. By analysing 452 commits from large, top-scoring JavaScript projects from GitHub, we found that flakiness caused by concurrency-related issues (e.g., async wait, race conditions or deadlocks) is the most dominant reason for test flakiness. The other top causes of flaky tests are operating system-specific (e.g., features that work on specific OS or OS versions) and network stability (e.g., internet availability or bad socket management). In terms of how flaky tests are dealt with, the majority of those flaky tests (>80%) are fixed to eliminate flaky behaviour and developers sometimes skip, quarantine or remove flaky tests.
