A Critical Evaluation of Spectrum-Based Fault Localization Techniques on a Large-Scale Software System
Here follows supplementary material for QRS 2017 reviewers.
Additional plots including all the SBFL ranking metrics considered for the experiments:
- summary-abslt-involved-zoom.pdf: Contains all bugs that were involved at least once (57 bugs) and shows the percentage of localized bugs for the first 4000 LOC.
- summary-abslt-involved.pdf: Contains all bugs that were involved at least once (57 bugs) and shows the percentage of localized bugs for the first 50000 LOC.
- summary-abslt-zoom.pdf: Contains all bugs (88 bugs) and shows the percentage of localized bugs for the first 4000 LOC.
- summary-abslt.pdf: Contains all bugs (88 bugs) and shows the percentage of localized bugs for the first 50000 LOC.
- summary-pctg-involved-zoom.pdf: Contains all bugs that were involved at least once (57 bugs) and shows the percentage of localized bugs when investigating the first 20% LOC of each buggy version.
- summary-pctg-involved.pdf: Contains all bugs that were involved at least once (57 bugs) and shows the percentage of localized bugs when investigating all LOC of each buggy version.
- summary-pctg-zoom.pdf: Contains all bugs (88 bugs) and shows the percentage of localized bugs when investigating the first 20% LOC of each buggy version.
- summary-pctg.pdf: Contains all bugs (88 bugs) and shows the percentage of localized bugs when investigating the first all LOC of each buggy version.
The whole dataset used for AspectJ, including the manual bug classification, coverage reports, and a Java library to read and manipulate them is available for download. Detailed reports for ANT, Lucene, and Rhino are available as well.
The iBugs benchmark suite, including AspectJ, can be found at https://www.st.cs.uni-saarland.de/ibugs