Practical Image/Video Quality
Metric
Fan ZHANG, Songnan LI, Lin MA (supervised by Profs. King Ngi NGAN, Weisi LIN, Wenyu
LIU)
History
When noting a difference
between things that have been compared,
we do not perceive the difference between
the things,
but the ratio of the difference to their magnitude.
----- Weber (1834)
The basic principle of photometry depicted by
Rubens in book Aguilonius (1613)
Category
|
|
FEATURE |
||||
|
a. Difference between the impaired and
the reference |
b. Deviance from the impaired to the
prior |
c. Impairment configurations in
processing chain of data |
d. Any heuristic features |
||
|
MODELING METHOD |
A. Casual modeling |
PSNR, SSIM, MS-SSIM, CW-SSIM, FISM, VIF, PSNR-HVS-M,VSNR, MAD, SEDLAI,
MOVIE, ABT-JND. |
–— |
–— |
|
|
B. Regression |
–— |
||||
|
C. Statistical learning |
, Q |
||||
Casual models _____ Descriptive models _____ Discriminative
models _____
Criteria
Ground truth of subjective quality databases
|
Image |
LIVE, IRCCyN-IVC, Toyama, A57, TID, WIQ, LAR, CSIQ; Fourier SB, BA, Meerwald |
|
Video |
IVP
HD, LIVE,
LIVE
wireless, VQEG
HD, EPFL-PolimI, NYU,
VQEG FRTV I, NTU |
Goodness-of-fitting criterion
|
RMSE |
Assumption of homoscedastic Gaussian
distribution of data |
|
Pearson correlation |
Invariant with linear transform (fitting) |
|
Spearman correlation |
Invariant with monotonic transform.
Applicable to ordinal data |
|
Kendall correlation |
Invariant with monotonic transform. Applicable to the rank pairs (even
breaking the rank transitivity) |
|
Outlier ratios |
Based on confidential interval of data |
|
Based on Gaussian distribution and confidential interval of data |
|
|
Deviance of likelihood [4] |
Adaptive to the prior distribution of data |
Our
Proposals
Our
proposals conform to the general form of distortion metric:
X and Y represent the reference
and the distorted images respectively. Image difference is calculated at first,
and then is decomposed into multiple channels, denoted by operator H. We use average filter [1], DCT [2],
and
KLT [3]. Finally, the image
difference in each channel is weighted by the diagonal matrix M. Several HVS related
features may be encapsulated into the weighting matrix M, we only employ texture masking
effect [1~3] and
contrast sensitive function (CSF) [2~3]. We learn the CSF associated with
DCT domain [2], while find that the
Eigen-values
are good CSF in KLT domain and happen to be opposite to Manhalanobis
distance [3]. The proposed
metrics are simple, fast to be optimized, and easy to be embedded into
perceptual processing system.
The above proposals aim at
ordinal match between the objective predictions and the subjective scores.
For
numerical match, it is necessary to monotonically map the objective
predictions toward the subjective scores.
Traditionally, metric design
and monotonic map are separated. We unified the two steps into a single
framework,
i.e..
Additive Log-logistic Model (ALM) framework.
Given totally M independent databases where the m-th database contain totally Nm samples, metric design is
posed as a maximum likelihood problem as
with a general log-logistic quality model
as
where feature vectors are x1, x2, …; sum
performs the spatial pooling, scale pooling, or impairments combination;
parameters {α,β,γ} are independent with databases while {a,b} are adaptive with each
database.
ALM has regularized procedures of feature selection and parameter estimation, and also supports co-training on
multiple databases. Following this methodology, we developed
1) a no-reference video
quality metric mainly based on the
impairment-relevant configurations (feature c) [4],
which
has won the model competition of
standardization work by ITU-T Study Group 12 in June, 2012.
2) a full-reference image quality metric
based on the feature a, which discloses the functional structure of V1 [5].
Publications
1.
F. Zhang, S.
Li, L. Ma, and K. N. Ngan, “Limitation and challenges of Image
Quality Measurement,”
presented
in SPIE Visual Communications and
Image Processing conference, 2010, pp. 1-8. [ code ] [ pdf ].
2.
F. Zhang, L. Ma,
S. Li, and K. N. Ngan, "Practical image quality metric
applied to image coding,"
IEEE Trans. Multimedia, vol. 13, pp. 615-624,
2011 (recommended by IEEE
MMTC R-letter). [ code ] [ pdf ]
3. F. Zhang, W. Liu, W. Lin, K. N. Ngan,
“Spread spectrum image watermarking based on perceptual quality metric,”
IEEE Trans. Image Processing, vol. 20,
pp. 3207-3218, 2011. [ code ] [ pdf ]
4.
F. Zhang, W. Lin, Z. Chen and K. N. Ngan, “Additive
log-logistic model for video quality assessment,”
IEEE Trans. Image Processing. vol. 22, pp. 1536-1547, 2013 (adopted by Standard ITU-T P.1202.2 Mode 2). [ pdf ]
5.
F. Zhang, W. Jiang, F. Autrusseau, W. Lin, “Exploring V1 by modeling the perceptual quality of images,”
Journal of Vision [ code (parameter estimation for Additive
Log-logisitic Model)] [pdf]
More
Resources
Third-party Matlab toolbox of existing metrics: MeTriX Mux VQA
Package
Dr. Stefan
Winkler’s link: Image and
video quality resources
Last update: Dec. 10, 2013