Practical Image/Video Quality
Metric
Fan ZHANG, Songnan LI, Lin MA (supervised by Profs. King Ngi NGAN, Weisi LIN, Wenyu
LIU)
History
When noting a difference
between things that have been compared,
we do not perceive the difference between
the things,
but the ratio of the difference to their magnitude.
 Weber (1834)
The basic principle of photometry depicted by
Rubens in book Aguilonius (1613)
Category

FEATURE 

a. Difference between the impaired and
the reference 
b. Deviance from the impaired to the
prior 
c. Impairment configurations in
processing chain of data 
d. Any heuristic features 

MODELING METHOD 
A. Casual modeling 
PSNR, SSIM, MSSSIM, CWSSIM, FISM, VIF, PSNRHVSM,VSNR, MAD, SEDLAI,
MOVIE, ABTJND. 
–— 
–— 

B. Regression 
–— 

C. Statistical learning 
, Q 
Casual models _____ Descriptive models _____ Discriminative
models _____
Criteria
Ground truth of subjective quality databases
Image 
LIVE, IRCCyNIVC, Toyama, A57, TID, WIQ, LAR, CSIQ; Fourier SB, BA, Meerwald 
Video 
IVP
HD, LIVE,
LIVE
wireless, VQEG
HD, EPFLPolimI, NYU,
VQEG FRTV I, NTU 
Goodnessoffitting criterion
RMSE 
Assumption of homoscedastic Gaussian
distribution of data 
Pearson correlation 
Invariant with linear transform (fitting) 
Spearman correlation 
Invariant with monotonic transform.
Applicable to ordinal data 
Kendall correlation 
Invariant with monotonic transform. Applicable to the rank pairs (even
breaking the rank transitivity) 
Outlier ratios 
Based on confidential interval of data 
Based on Gaussian distribution and confidential interval of data 

Deviance of likelihood [4] 
Adaptive to the prior distribution of data 
Our
Proposals
Our
proposals conform to the general form of distortion metric:
X and Y represent the reference
and the distorted images respectively. Image difference is calculated at first,
and then is decomposed into multiple channels, denoted by operator H. We use average filter [1], DCT [2],
and
KLT [3]. Finally, the image
difference in each channel is weighted by the diagonal matrix M. Several HVS related
features may be encapsulated into the weighting matrix M, we only employ texture masking
effect [1~3] and
contrast sensitive function (CSF) [2~3]. We learn the CSF associated with
DCT domain [2], while find that the
Eigenvalues
are good CSF in KLT domain and happen to be opposite to Manhalanobis
distance [3]. The proposed
metrics are simple, fast to be optimized, and easy to be embedded into
perceptual processing system.
The above proposals aim at
ordinal match between the objective predictions and the subjective scores.
For
numerical match, it is necessary to monotonically map the objective
predictions toward the subjective scores.
Traditionally, metric design
and monotonic map are separated. We unified the two steps into a single
framework,
i.e..
Additive Loglogistic Model (ALM) framework.
Given totally M independent databases where the mth database contain totally Nm samples, metric design is
posed as a maximum likelihood problem as
with a general loglogistic quality model
as
where feature vectors are x1, x2, …; sum
performs the spatial pooling, scale pooling, or impairments combination;
parameters {α,β,γ} are independent with databases while {a,b} are adaptive with each
database.
ALM has regularized procedures of feature selection and parameter estimation, and also supports cotraining on
multiple databases. Following this methodology, we developed
1) a noreference video
quality metric mainly based on the
impairmentrelevant configurations (feature c) [4],
which
has won the model competition of
standardization work by ITUT Study Group 12 in June, 2012.
2) a fullreference image quality metric
based on the feature a, which discloses the functional structure of V1 [5].
Publications
1.
F. Zhang, S.
Li, L. Ma, and K. N. Ngan, “Limitation and challenges of Image
Quality Measurement,”
presented
in SPIE Visual Communications and
Image Processing conference, 2010, pp. 18. [ code ] [ pdf ].
2.
F. Zhang, L. Ma,
S. Li, and K. N. Ngan, "Practical image quality metric
applied to image coding,"
IEEE Trans. Multimedia, vol. 13, pp. 615624,
2011 (recommended by IEEE
MMTC Rletter). [ code ] [ pdf ]
3. F. Zhang, W. Liu, W. Lin, K. N. Ngan,
“Spread spectrum image watermarking based on perceptual quality metric,”
IEEE Trans. Image Processing, vol. 20,
pp. 32073218, 2011. [ code ] [ pdf ]
4.
F. Zhang, W. Lin, Z. Chen and K. N. Ngan, “Additive
loglogistic model for video quality assessment,”
IEEE Trans. Image Processing. vol. 22, pp. 15361547, 2013 (adopted by Standard ITUT P.1202.2 Mode 2). [ pdf ]
5.
F. Zhang, W. Jiang, F. Autrusseau, W. Lin, “Exploring V1 by modeling the perceptual quality of images,”
Journal of Vision [ code (parameter estimation for Additive
Loglogisitic Model)] [pdf]
More
Resources
Thirdparty Matlab toolbox of existing metrics: MeTriX Mux VQA
Package
Dr. Stefan
Winkler’s link: Image and
video quality resources
Last update: Dec. 10, 2013