Fix fixed-pt PVQ compand/expand outputs zero gain
For fixed-point version of PVQ, which is current default, added MAXI(1, ) to limit the minimum companded or expanded gain to be one. Previously, gain compand/expand function, which is invoked when activity masking is enabled, sometimes outputs zero then triggered the assert(gain != 0). Metric change from floating-pt to fixed-pt PVQ is: PSNR PSNR-HVS SSIM CIEDE-2000 PSNR Cb PSNR Cr MS-SSIM VMAF 0.02 0.10 0.08 0.11 0.01 0.02 0.13 -0.30 Change-Id: I64a60d1970d35a26af227841e4a5e50a89ddc44c
Loading
Please register or sign in to comment