Designing better fitness functions for automated program repair

Ethan Fast, Claire Le Goues, Stephanie Forrest, Westley Weimer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

35 Citations (Scopus)

Abstract

Evolutionary methods have been used to repair programs automatically, with promising results. However, the fitness function used to achieve these results was based on a few simple test cases and is likely too simplistic for larger programs and more complex bugs. We focus here on two aspects of fitness evaluation: efficiency and precision. Efficiency is an issue because many programs have hundreds of test cases, and it is costly to run each test on every individual in the population. Moreover, the precision of fitness functions based on test cases is limited by the fact that a program either passes a test case, or does not, which leads to a fitness function that can take on only a few distinct values. This paper investigates two approaches to enhancing fitness functions for program repair, incorporating (1) test suite selection to improve efficiency and (2) formal specifications to improve precision. We evaluate test suite selection on 10 programs, improving running time for automated repair by 81%. We evaluate program invariants using the Fitness Distance Correlation (FDC) metric, demonstrating significant improvements and smoother evolution of repairs.

Original languageEnglish (US)
Title of host publicationProceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10
Pages965-972
Number of pages8
DOIs
StatePublished - Aug 27 2010
Externally publishedYes
Event12th Annual Genetic and Evolutionary Computation Conference, GECCO-2010 - Portland, OR, United States
Duration: Jul 7 2010Jul 11 2010

Other

Other12th Annual Genetic and Evolutionary Computation Conference, GECCO-2010
CountryUnited States
CityPortland, OR
Period7/7/107/11/10

Fingerprint

Fitness Function
Repair
Fitness
Efficiency Evaluation
Evaluate
Formal Specification
Likely
Distinct
Metric
Invariant

Keywords

  • Genetic programming
  • Software engineering
  • Software repair

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Theoretical Computer Science

Cite this

Fast, E., Goues, C. L., Forrest, S., & Weimer, W. (2010). Designing better fitness functions for automated program repair. In Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10 (pp. 965-972) https://doi.org/10.1145/1830483.1830654

Designing better fitness functions for automated program repair. / Fast, Ethan; Goues, Claire Le; Forrest, Stephanie; Weimer, Westley.

Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10. 2010. p. 965-972.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fast, E, Goues, CL, Forrest, S & Weimer, W 2010, Designing better fitness functions for automated program repair. in Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10. pp. 965-972, 12th Annual Genetic and Evolutionary Computation Conference, GECCO-2010, Portland, OR, United States, 7/7/10. https://doi.org/10.1145/1830483.1830654
Fast E, Goues CL, Forrest S, Weimer W. Designing better fitness functions for automated program repair. In Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10. 2010. p. 965-972 https://doi.org/10.1145/1830483.1830654
Fast, Ethan ; Goues, Claire Le ; Forrest, Stephanie ; Weimer, Westley. / Designing better fitness functions for automated program repair. Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10. 2010. pp. 965-972
@inproceedings{5140dc13009d405d947c16f6aee3812a,
title = "Designing better fitness functions for automated program repair",
abstract = "Evolutionary methods have been used to repair programs automatically, with promising results. However, the fitness function used to achieve these results was based on a few simple test cases and is likely too simplistic for larger programs and more complex bugs. We focus here on two aspects of fitness evaluation: efficiency and precision. Efficiency is an issue because many programs have hundreds of test cases, and it is costly to run each test on every individual in the population. Moreover, the precision of fitness functions based on test cases is limited by the fact that a program either passes a test case, or does not, which leads to a fitness function that can take on only a few distinct values. This paper investigates two approaches to enhancing fitness functions for program repair, incorporating (1) test suite selection to improve efficiency and (2) formal specifications to improve precision. We evaluate test suite selection on 10 programs, improving running time for automated repair by 81{\%}. We evaluate program invariants using the Fitness Distance Correlation (FDC) metric, demonstrating significant improvements and smoother evolution of repairs.",
keywords = "Genetic programming, Software engineering, Software repair",
author = "Ethan Fast and Goues, {Claire Le} and Stephanie Forrest and Westley Weimer",
year = "2010",
month = "8",
day = "27",
doi = "10.1145/1830483.1830654",
language = "English (US)",
isbn = "9781450300728",
pages = "965--972",
booktitle = "Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10",

}

TY - GEN

T1 - Designing better fitness functions for automated program repair

AU - Fast, Ethan

AU - Goues, Claire Le

AU - Forrest, Stephanie

AU - Weimer, Westley

PY - 2010/8/27

Y1 - 2010/8/27

N2 - Evolutionary methods have been used to repair programs automatically, with promising results. However, the fitness function used to achieve these results was based on a few simple test cases and is likely too simplistic for larger programs and more complex bugs. We focus here on two aspects of fitness evaluation: efficiency and precision. Efficiency is an issue because many programs have hundreds of test cases, and it is costly to run each test on every individual in the population. Moreover, the precision of fitness functions based on test cases is limited by the fact that a program either passes a test case, or does not, which leads to a fitness function that can take on only a few distinct values. This paper investigates two approaches to enhancing fitness functions for program repair, incorporating (1) test suite selection to improve efficiency and (2) formal specifications to improve precision. We evaluate test suite selection on 10 programs, improving running time for automated repair by 81%. We evaluate program invariants using the Fitness Distance Correlation (FDC) metric, demonstrating significant improvements and smoother evolution of repairs.

AB - Evolutionary methods have been used to repair programs automatically, with promising results. However, the fitness function used to achieve these results was based on a few simple test cases and is likely too simplistic for larger programs and more complex bugs. We focus here on two aspects of fitness evaluation: efficiency and precision. Efficiency is an issue because many programs have hundreds of test cases, and it is costly to run each test on every individual in the population. Moreover, the precision of fitness functions based on test cases is limited by the fact that a program either passes a test case, or does not, which leads to a fitness function that can take on only a few distinct values. This paper investigates two approaches to enhancing fitness functions for program repair, incorporating (1) test suite selection to improve efficiency and (2) formal specifications to improve precision. We evaluate test suite selection on 10 programs, improving running time for automated repair by 81%. We evaluate program invariants using the Fitness Distance Correlation (FDC) metric, demonstrating significant improvements and smoother evolution of repairs.

KW - Genetic programming

KW - Software engineering

KW - Software repair

UR - http://www.scopus.com/inward/record.url?scp=77955916560&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77955916560&partnerID=8YFLogxK

U2 - 10.1145/1830483.1830654

DO - 10.1145/1830483.1830654

M3 - Conference contribution

AN - SCOPUS:77955916560

SN - 9781450300728

SP - 965

EP - 972

BT - Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10

ER -