A crucial operation in image and video processing applications is affine transforms. Typical applications of affine transforms include fractal block coding, camera operation detection, affine motion estimation, etc. Affine transforms involve complex operations and are hence difficult to implement in real-time. In this paper, we present a novel architecture for real-time implementation of affine transforms. First, we derive two fundamental operations from affine transforms and then propose an efficient method of implementing these operations. As an example of the application of ATP (Affine Transform Processor), we propose a high performance video compression algorithm mapped onto the proposed architecture. This algorithm is based on combined affine transform and vector quantization (ATVQ), where the infra-frame and inter-frame redundancy in the video sequence are exploited through piecewise self-similarity on a block-wise basis within a frame and between frames. ATVQ has the advantages of superior coding performance at a significantly reduced computational complexity. ATVQ has been mapped onto the ATP and real-time execution is demonstrated using a VHDL (VHSIC Hardware Description Language) implementation of ATP.