Design of integrated very low intermediate frequency (VLIF) RF receivers with better than 35dB image rejection (IR) usually requires detection and calibration of I/Q channel mismatches less than 0.1dB in amplitude and 1.8 degree in phase. A mismatch amplification technique for improving detection sensitivity is presented in this paper. The proposed mismatch transfer function technique provides an effective method to calculate internal amplitude and phase mismatches from the measured EVM at the receiver output. Experimental data from an integrated VLIF receiver test chip is used to demonstrate the theory. By utilizing this technique, an I/Q mismatch measurement resolution as low as 0.01 dB in amplitude and 0.18 degree in phase is achieved.