• Yushin Cho's avatar
    Fix fixed-pt PVQ compand/expand outputs zero gain · 3ebfe2a4
    Yushin Cho authored
    For fixed-point version of PVQ, which is current default,
    added MAXI(1, ) to limit the minimum companded or expanded gain to be one.
    Previously, gain compand/expand function, which is invoked when
    activity masking is enabled, sometimes outputs zero
    then triggered the assert(gain != 0).
    
    Metric change from floating-pt to fixed-pt PVQ is:
    PSNR  PSNR-HVS  SSIM  CIEDE-2000  PSNR Cb PSNR Cr MS-SSIM VMAF
    0.02  0.10      0.08  0.11        0.01    0.02    0.13    -0.30
    
    Change-Id: I64a60d1970d35a26af227841e4a5e50a89ddc44c
    3ebfe2a4
pvq.c 33.8 KB