Peening process is a widely used method to improve fatigue strength of metals. It has been shown that the increased fatigue performance is mainly due to compressive residual stress near specimen surface during peening processes. However, the damage tolerance analysis directly using the residual stress profile after manufacturing will have a very non-conservative prediction results. A detailed mechanism model is proposed to capture the physics of fatigue damage accumulation of shot peened specimens. The proposed methodology is based on the Equivalent Initial Flaw Size (EIFS) concept and the asymptotic stress intensity factor solution of notched specimen. Two detrimental effects of the fatigue behavior of shot peened specimens are included into the proposed methodology. One is the residual stress relaxation and the other is the increased surface roughness. Time varying residual stress and the surface notch depth (roughness) are included in the EIFS life prediction methodology for damage tolerance analysis. A simplified probabilistic damage tolerance analysis model is proposed. The developed probabilistic damage tolerance analysis methodology is compared with various experimental data under different shot peening conditions and very good agreement is observed between model predictions and experimental data.