• Timothy B. Terriberry's avatar
    Scale PVQ input to OD_COEFF_SHIFT resolution. · e93acb2d
    Timothy B. Terriberry authored
    This ensures we operate at the same precision that Daala uses, which matters
    when activity masking is enabled, because of the gain companding.
    
    Metrics from Patchset 4 (which had slightly incorrect rounding):
    
    With activity masking (5 frames only):
    av1_pvq_AM_ref_5f@2017-02-07T03:37:53.702Z -> av1_pvq_AM_derf_fix2_coeff_scaling_5f@2017-02-07T00:12:24.427Z
    
        PSNR |  PSNR Cb |  PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.6653 | -12.3177 | -12.1858 |   0.3350 | 4.1013 |  2.0964 |    -4.0539
    
    In particular for Netflix_Crosswalk_1920x1080_60fps_8bit_420_60f.y4m
     -5.0589 | -22.3077 | -21.2188 |  -7.0389 | -3.3715 |-5.7794 |   -13.1891
    
    I.e., it fixes the large regression with AM on this sequence, and
     substantially improves chroma (at a lesser cost to other metrics).
    
    Without activity masking (5 frames only):
    av1_pvq_ref_5f@2017-02-07T03:52:51.279Z -> av1_pvq_derf_fix2_coeff_scaling_5f@2017-02-07T00:12:48.873Z
    
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0989 | -0.0322 | -0.0464 |   0.1883 | 0.0795 |  0.0579 |     0.0923
    
    Change-Id: I46b808b7c8e4733465f8bebc8336dfd5b75783ec
    e93acb2d
decodeframe.c 186 KB