Sparse representations using predefined and learned dictionaries have widespread applications in signal and image processing. Sparse approximation techniques can be used to recover data from its low dimensional corrupted observations, based on the knowledge that the data is sparsely representable using a known dictionary. In this paper, we propose a method to improve data recovery by ensuring that the data recovered using sparse approximation is close its manifold. This is achieved by performing regularization using examples from the data manifold. This technique is particularly useful when the observations are highly reduced in dimensions when compared to the data and corrupted with high noise. Using an example application of image inpainting, we demonstrate that the proposed algorithm achieves a reduction in reconstruction error in comparison to using only sparse coding with predefined and learned dictionaries, when the percentage of missing pixels is high.