Mining plausible hypotheses from the literature via meta-analysis

Abstract

Meta-analysis is highly advocated in many fields of empirical research such as medicine and psychology, mainly due to its capability to synthesize quantitative evidence of effects from the literature, based on statistical analysis. However, the adoption of meta-analysis to software engineering is still suffering from inertia, despite the fact that many software engineering researchers have long been arguing the need for it. As an attempt to move beyond the lockstep, we in this paper explore a different use of meta-analysis. That is, our proposition is that meta-analysis is useful for mining hypotheses because their plausibility is backed by evidence accumulated in the literature, and thus researchers could focus their effort on the areas that are of particular need. We assess our proposition by conducting a lightweight case study on the literature of defect prediction. We found that three out of five hypotheses we extract from our meta-analysis were indeed investigated in separate papers, indicating the usefulness of our approach. We also recognize two uninvestigated hypotheses whose validity we plan to investigate in the future.