Background: The increasing complexity of software systems necessitates robust and efficient testing methods. While Search-Based Software Testing (SBST) has emerged as a powerful technique for automated test case generation, its effectiveness can be limited by its singular focus on code coverage. The generated tests, although structurally sound, may not target the most fault-prone areas of the code.
Aim: This study aims to address this limitation by proposing and empirically investigating a novel approach that integrates defect prediction (DP) models to guide the search process of SBST. By leveraging insights from historical code data, our method prioritizes the generation of test cases for code modules identified as having a higher likelihood of containing defects.
Method: We conducted a large-scale empirical study using 20 real-world, open-source Java projects from the Defects4J database. We developed a machine learning-based defect prediction model to identify fault-prone files. We then implemented a new fitness function for the EvoSuite test generation tool that incorporates the prediction score. The performance of this defect prediction-guided SBST approach was compared against a traditional, coverage-based SBST approach, using metrics of fault detection effectiveness and computational efficiency.
Results: Our findings indicate that the proposed DP-guided SBST approach significantly outperforms the traditional method in terms of the number of unique faults detected. Statistical analysis revealed a strong positive effect size for our approach. While there was a slight increase in computational overhead associated with the defect prediction component, it was minimal relative to the substantial gain in fault detection.
Conclusion: The results demonstrate that integrating defect prediction into the search-based test generation process is a highly effective strategy for improving the overall quality and fault-finding capability of automated testing. This approach represents a promising direction for enhancing software testing practices, particularly in continuous integration environments.
