The quantification of variability in the mechanical behavior of metallic materials is important in the design and reliability assessment of mechanical components. A combination of experimental and computational approaches is often required to alleviate the experimental burden and lack of data in constructing a probabilistic formalism for material design. The present work aims at integrating material characterization and computational modeling for the evaluation of variability in the elastodynamic response of random polycrystals. First, a procedure is presented for simulation of random 2D polycrystalline microstructures from limited experimental data. Second, the capability of the numerical model in capturing the variation of the scattered waves due to the random heterogeneities is investigated by introducing a suitable quantity of interest characterizing the intensity of the fluctuations of the stochastic waveforms. Two important types of heterogeneities are considered. The first is the inherent heterogeneity due to the mismatch in the grain orientations. The second is the heterogeneity due to fine scale defects in the form of random intergranular micro-cavities. The numerical model presented in this paper can be useful for the interpretation of experimental ultrasonic measurements for random heterogeneous material. The result is also applicable to the validation of multiscale probabilistic models for material prognosis.