A multi-transform search algorithms to enhance H.264/AVC performance

We propose to extend the basic H.264/AVC by the adding several linear transformations in the transform phase of the encoder [2]. The proposed extension improves the performance of the video compression. We define a criteria and an algorithm that identifies several linear transformations that improve the output of the AVC encoder. First, we analyze the Discrete Cosine Transform (DCT). If it fails to provide satisfactory compressed output we provide additional transforms to replace it. We consider the standard H264/AVC as the basic scheme to which we add the transforms. Each added transform that replaces the DCT should enhance the quality of the encoder output. We use training data that is extracted from applying the H.264/AVC encoder on real video data sets. The algorithm divides the training data into several subgroups. Then a transform is found to each sub-group. The transform search is performed offline on a large-scale residual data that was created from several movies. Although these transforms fit specific video contents they also provide good performance on different contents while maintaining similar compression gain. By modifying linear transformation instead of a dividing the data into several subgroups a significant complexity reduction is achieved. The minimization of cost function splits the residual data into two groups. The process is repeated recursively to obtain additional transforms. This coding process provides the encoder with a freedom to choose for each block of data, which transform to use from an available collection of transforms that provide significant compression gain when using the Rate Distortion Optimization algorithm developed in [3]. By adding three transforms to the Integer-Transform (DCT version of the H.264/AVC) the required bit-rate for inter-coding of residual data is reduced by up to 30% without any degradation in the PSNR. The proposed method can also be used in other block-based coding schemes.