Human visual cortex is organized into several functional re-gions/areas. Identifying these visual areas of the human brain (i.e., V1, V2, V4, etc) is an important topic in neurophysiology and vision science. Retinotopic mapping via functional magnetic resonance imaging (fMRI) provides a noninvasive way of defining the boundaries of the visual areas. It is well known from neurophysiology studies that retino-topic mapping is diffeomorphic within each local area (i.e. locally smooth, differentiable, and invertible). However, due to the low signal-noise ratio of fMRI, the retinotopic maps from fMRI are often not diffeomorphic, making it difficult to delineate the boundaries of visual areas. The purpose of this work is to generate diffeomorphic retinotopic maps and improve the accuracy of the retinotopic atlas from fMRI measurements through the development of a specifically designed registration procedure. Although there are sophisticated existing cortical surface registration methods, most of them cannot fully utilize the features of retinotopic mapping. By considering unique retinotopic mapping features, we form a qua-siconformal geometry-based registration model and solve it with efficient numerical methods. We compare our registration with several popular methods on synthetic data. The results demonstrate that the proposed registration is superior to conventional methods for the registration of retinotopic maps. The application of our method to a real retinotopic mapping dataset also results in much smaller registration errors.