Filtering Out Incorrect Patches with Generalized Tests

Abstract

Automated program repairing has been growing rapidly in the last years. There are many approaches for using these techniques and one of them is generate-and-validate. Generate-and-validate or, G&V has become very popular. When the APR tool generates patches, overfitting patches are also included there. That means, those patches are actually not correct patches but they pass all the tests available in the test suite. This overfitting problem is a big challenge in APR still now. So the goal of the G&V approach is eliminating the overfitting patches after patches are generated by the APR tool. This work introduces a novel lightweight specification methodology that uses an existing failing test. So the idea is that if developers generalize the existing failing tests by expressing their insight about the bug, then many incorrect patches (generated by the APR tools) in the list can be filtered out and the user has to review only the remaining patches. Our approach can be particularly useful when an APR tool returns a list of plausible patches to the user, as done in multiple recent APR tools. Many incorrect patches in the list can be filtered out using a single generalized test, and the user only needs to review the remaining patches. Our additional experiment results on a state-of-the-art APR tool, JAID, show that this usage scenario is promising, filtering out hundreds of incorrect patches while keeping all correct patches.