High Performance Loop Filtering in Video Compression 

Introduction

        Coding artifacts appear in coded images and video sequences. Non-local Kuan¡¯s (NLK) filter has achieved excellent performance to suppress blocking and ringing artifacts on images coded using transform coding such as JPEG. However, direct application of the NLK filter on videos coded using intra/inter-frame prediction and transform coding cannot improve coding performance efficiently. We identify the causes of the problem and propose quadtree-based NLK (QNLK) loop filter and quadtree-based overcomplete NLK (QOCNLK) loop filter to solve the problem. NLK and overcomplete NLK loop filters are used to restore quantized residual transform coefficients. Restored coefficients are then projected onto designed quantization constraint sets (QCS). Quadtree-based signaling strategy is used for adaptive filtering control. Experimental results show that the proposed loop filtering techniques achieve significant bit rate saving and visual quality improvement compared with H.264 or advanced video coding (AVC) High Profile.

The NLK Loop Filter
        The non-local Kuan's filter was proposed in [1]. The detailed derivation and discussion of the NLK filter were described in [1].

        A pilot frame is used to estimate parameters, mean and variance, and then the NLK filter is performed. The state-of-the-art QALF+DLF loop filter [2] is used to generated pilot frames. The following figure shows the block diagram of the NLK loop filter.

¡¡

The Overcomplete NLK Loop Filter

        The overcomplete transform is superior to non-overcomplete representation in many image processing applications. It provides redundant estimates of the same image pixels. Averaging of the estimates for the same image pixels can produce better performance.An overcomplete NLK loop filter for video data is shown below.

¡¡

Quadtree Technique
        A block-based technique is used to selectively apply the NLK and OCNLK loop filters:

Im,n is the block control, which is either 1 or 0. Block control information indicates the NLK or OCNLK filters ON-OFF for each block. It is decided by rate-distortion optimization selection

D is the distortion between a processed frame and its original frame;  R is the bit rate for block control side information. These block controls are coded by CABAC as side information and transmitted to the decoder. The decoder then determines whether to use the proposed NLK filter or not after decoding the transmitted block controls. The NLK or OCNLK filter is used for blocks selected by the block control information Im,n coded using quadtree structure. Hence, we name the proposed techniques quadtree-based NLK (QNLK) loop filter and quadtree-based OCNLK (QOCNLK) loop filter.

Experimental Results

        1. The table below shows Bjontegaard delta bit rate (¦¤BR) of CIF and 720p coded sequences by QALF+DLF, QNLK and QOCNLK methods. It can be observed in the following experimental results that  

            QNLK and QOCNLK methods all perform significantly better than others.

  Bit rate reduction ¦¤BR(%) of cif by three loop filtering methods, compared with H.264/AVC high profile

CIF Sequence

(352x288)

QALF+DLF

QNLK Loop

QOCNLK Loop

IPPP

IBBP

IPPP

IBBP

IPPP

IBBP

foreman

-5.36

-4.18

-7.3

-5.99

-8.64

-7.25

carphone

-3.85

-3.88

-6.65

-6.31

-8.61

-7.99

bus

-3.95

-3.67

-4.81

-4.57

-5.55

-5.3

football

-8.14

-7.12

-9.43

-7.65

-10.69

-8.9

container

-7.69

-6.71

-8.72

-7.61

-9.62

-8.54

mother_daughter

-5.25

-4.37

-6.96

-6.37

-9.11

-8.41

Average

-5.71

-4.99

-7.31

-6.42

-8.70

-7.73

 Bit rate reduction ¦¤BR(%) of 720p by three loop filtering methods, compared with H.264/AVC high profile

720p Sequence

(1280x720)

QALF+DLF

QNLK Loop

QOCNLK Loop

IPPP

IBBP

IPPP

IBBP

IPPP

IBBP

Sheriff

-7.95

-7.74

-9.97

-10.09

-11.24

-11.22

ShuttleStart

-9.5

-9.8

-10.77

-11.24

-12.03

-11.87

Night

-6.92

-6.08

-8.97

-8.26

-10.72

-9.74

Optis

-8.7

-8.19

-9.7

-10.02

-10.71

-11.26

Sailormen

-8.6

-8.59

-11.51

-11.59

-12.59

-12.77

Spincalendar

-8.51

-7.67

-11.69

-11.11

-13

-12.62

Average

-8.36

-8.01

-10.44

-10.39

-11.72

-11.58

         2. The figure below shows the R-D curves for the CIF sequence ¡°Mother_daughter¡±. The proposed two methods consistently perform well at any coding quality setting.

           3. QNLK and QOCNLK filtered frames have better visual quality than QALF+DLF and H.264 coded frames. In the 720p sequence ¡°Spincalendar¡±, letters are restored better with fewer artifacts by our

               methods. QOCNLK filter slightly performs better than QNLK method.

(a) Original frame

   

(b) H.264 coded frame                                                 (c) QALF+DLF filtered frame

  

(d) QNLK filtered frame                                                    (e) QOCNLK filtered frame

¡¡

Demo Program

An executable demo program can be downloaded here.  This program can be run under MS Visual Studio 2005. 

     Code
   Instruction
References
[1] R. Zhang, W. Ouyang, W.K. Cham,, ¡°Image Postprocessing by Non-local Kuan¡¯s Filter,¡± Journal of Visual Communication and Image Representation, Elsevier, 22 (2011) 251¨C262.

[2] T. Chujoh, N. Wada, and G. Yasuda, ¡°Quadtree-based adaptive loop filter,¡± ITU-T SG16 Contribution, C181, Geneva, Jan. 2009.

[3] P. List, A. Joch, J. Lainema, G. Bjontegaard, M. Karczewicz, ¡°Adaptive deblocking filter,¡± IEEE Trans. Circuits Syst. Video Technol., 13(7), July 2003.

[4] Renqi Zhang Wai-kuen Cham and Yu Liu, ``High Performance Loop Filters in Video compression,'' submitted to IEEE Trans. Circuits and Systems for video Technology.

[5] Wai-Kuen Cham and Renqi Zhang, "High Performance Loop Filters in Video Compression", US Non-Provisional Patent Application Number 12/984,464, filed on January 4, 2011.

Contacts:

Mr. Renqi ZHANG, Prof. Wai-Kuen Cham

Email:  rqzhang@ee.cuhk.edu.hk  wkcham@ee.cuhk.edu.hk    

¡¡

¡¡