Digging into Semantics: Where Do Search-Based Software Repair Methods Search?

Hammad Ahmad, Padriac Cashin, Stephanie Forrest, Westley Weimer

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Search-based methods are a popular approach for automatically repairing software bugs, a field known as automated program repair (APR). There is increasing interest in empirical evaluation and comparison of different APR methods, typically measured as the rate of successful repairs on benchmark sets of buggy programs. Such evaluations, however, fail to explain why some approaches succeed and others fail. Because these methods typically use syntactic representations, i.e., source code, we know little about how the different methods explore their semantic spaces, which is relevant for assessing repair quality and understanding search dynamics. We propose an automated method based on program semantics, which provides quantitative and qualitative information about different APR search-based techniques. Our approach requires no manual annotation and produces both mathematical and human-understandable insights. In an empirical evaluation of 4 APR tools and 34 defects, we investigate the relationship between search-space exploration, semantic diversity and repair success, examining both the overall picture and how the tools’ search unfolds. Our results suggest that population diversity alone is not sufficient for finding repairs, and that searching in the right place is more important than searching broadly, highlighting future directions for the research community.

Original languageEnglish (US)
Title of host publicationParallel Problem Solving from Nature – PPSN XVII - 17th International Conference, PPSN 2022, Proceedings
EditorsGünter Rudolph, Anna V. Kononova, Hernán Aguirre, Pascal Kerschke, Gabriela Ochoa, Tea Tušar
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages16
ISBN (Print)9783031147203
StatePublished - 2022
Externally publishedYes
Event17th International Conference on Parallel Problem Solving from Nature, PPSN 2022 - Dortmund, Germany
Duration: Sep 10 2022Sep 14 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13399 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference17th International Conference on Parallel Problem Solving from Nature, PPSN 2022


  • Patch diversity
  • Program repair
  • Semantic search spaces

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Digging into Semantics: Where Do Search-Based Software Repair Methods Search?'. Together they form a unique fingerprint.

Cite this