Commit e93acb2d authored by Committed by Tim TerriberryBrowse files
Scale PVQ input to OD_COEFF_SHIFT resolution.
This ensures we operate at the same precision that Daala uses, which matters when activity masking is enabled, because of the gain companding. Metrics from Patchset 4 (which had slightly incorrect rounding): With activity masking (5 frames only): av1_pvq_AM_ref_5f@2017-02-07T03:37:53.702Z -> av1_pvq_AM_derf_fix2_coeff_scaling_5f@2017-02-07T00:12:24.427Z PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | CIEDE 2000 0.6653 | -12.3177 | -12.1858 | 0.3350 | 4.1013 | 2.0964 | -4.0539 In particular for Netflix_Crosswalk_1920x1080_60fps_8bit_420_60f.y4m -5.0589 | -22.3077 | -21.2188 | -7.0389 | -3.3715 |-5.7794 | -13.1891 I.e., it fixes the large regression with AM on this sequence, and substantially improves chroma (at a lesser cost to other metrics). Without activity masking (5 frames only): av1_pvq_ref_5f@2017-02-07T03:52:51.279Z -> av1_pvq_derf_fix2_coeff_scaling_5f@2017-02-07T00:12:48.873Z PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | CIEDE 2000 0.0989 | -0.0322 | -0.0464 | 0.1883 | 0.0795 | 0.0579 | 0.0923 Change-Id: I46b808b7c8e4733465f8bebc8336dfd5b75783ec