Evaluating Regression Test Selection Opportunities in a Very Large Open-Source Ecosystem

摘要: Regression testing in very large software ecosystems is notoriously costly, requiring computational resources that even corporations struggle to cope with. Very contain thousands of rapidly evolving, interconnected projects where client transitively depend on library projects. test selection (RTS) reduces regression costs by rerunning only tests whose pass/fail behavior may flip after code changes. For single projects, researchers showed class-level RTS more effective than lower method-or statement-level RTS. Meanwhile, several industry, e.g., at Facebook, Google, and Microsoft, perform project-level RTS, a changed all its transitive clients. However, there was no previous study the comparative benefits such ecosystems. We evaluate opportunities MAVEN Central opensource ecosystem. There, some popular libraries have up 924589 clients; turn, clients can 11190 libraries. sampled 408 found 202 (almost half) cannot update latest versions without breaking compilation or tests. If developers want detect these breakages earlier, they need run many compared four variants with Central. The results be an order magnitude less costly Specifically, various select, average, 7.8%-17.4% selected