A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each

Claire Le Goues, Michael Dewey-Vogt, Stephanie Forrest, Westley Weimer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

335 Scopus citations

Abstract

There are more bugs in real-world programs than human programmers can realistically address. This paper evaluates two research questions: "What fraction of bugs can be repaired automatically?" and "How much does it cost to repair a bug automatically?" In previous work, we presented GenProg, which uses genetic programming to repair defects in off-the-shelf C programs. To answer these questions, we: (1) propose novel algorithmic improvements to GenProg that allow it to scale to large programs and find repairs 68% more often, (2) exploit GenProg's inherent parallelism using cloud computing resources to provide grounded, human-competitive cost measurements, and (3) generate a large, indicative benchmark set to use for systematic evaluations. We evaluate GenProg on 105 defects from 8 open-source programs totaling 5.1 million lines of code and involving 10,193 test cases. GenProg automatically repairs 55 of those 105 defects. To our knowledge, this evaluation is the largest available of its kind, and is often two orders of magnitude larger than previous work in terms of code or test suite size or defect count. Public cloud computing prices allow our 105 runs to be reproduced for $403; a successful repair completes in 96 minutes and costs $7.32, on average.

Original languageEnglish (US)
Title of host publicationProceedings - 34th International Conference on Software Engineering, ICSE 2012
Pages3-13
Number of pages11
DOIs
StatePublished - Jul 30 2012
Externally publishedYes
Event34th International Conference on Software Engineering, ICSE 2012 - Zurich, Switzerland
Duration: Jun 2 2012Jun 9 2012

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257

Other

Other34th International Conference on Software Engineering, ICSE 2012
CountrySwitzerland
CityZurich
Period6/2/126/9/12

Keywords

  • automated program repair
  • cloud computing
  • genetic programming

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each'. Together they form a unique fingerprint.

Cite this