Water distribution networks (WDNs) are vital in transporting water to consumers as per demands. Traditionally, cost has been the main objective in the design of WDNs; however, minimizing cost may result in a system that is incapable of handling abnormal states such as demand uncertainties and pipe failures. Subsequently, various reliability and its surrogate measures (such as resilience metrics) were developed in the past for designing networks capable of handling uncertainties. Few studies exist that evaluate the performance of such resilience metrics in the context of mechanical failures such as water main breaks. This paper analyzes the performance of three resilience metrics: 1) Resilience Index (RI); 2) Network Resilience Index (NRI); and 3) Modified Resilience Index (MRI). Two popular benchmark WDN problems are considered in this research: a) A small Greenfield scenario and b) Expansion of an existing WDN to meet future demands. A genetic algorithm (GA)-based, multi-objective optimization formulation is employed for the design of WDNs by considering two objectives: a) Cost and b) Resilience. All single and double pipe breaks are considered as possible failure states for the performance evaluation of the obtained design solutions. The results from the analysis on limited number of benchmark networks indicate that NRI is a reasonable metric that is capable of guiding the design of resilient WDNs. Given the state of our underground water infrastructure, this research is very relevant and the results are immensely helpful in instilling resilience into existing or new water distribution networks.