1. 20 Nov, 2017 1 commit
    • Monty Montgomery's avatar
      Enable configurable fixed-depth coefficients in Daala TX · 57f6bfd0
      Monty Montgomery authored
      This patch turns on the fixed-depth TX code in the Daala toplevel TX
      
      A REGRESSION IS EXPECTED as this is temporarily dropping Daala TX back
      to Q3, which is reduced operating precision over current master.
      
      subset-1:
      monty-rest-of-stack-RDO-s1@2017-11-13T14:39:17.093Z ->
       monty-rest-of-stack-Q3-s1@2017-11-13T14:39:52.160Z
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0927 |     N/A | -0.0001 |   0.1390 | 0.0871 |  0.0835 |     0.0826
      
      objective-1-fast --limit=4:
      monty-rest-of-stack-RDO-o1f4@2017-11-13T14:38:57.951Z ->
       monty-rest-of-stack-Q3-o1f4@2017-11-13T14:39:32.205Z
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0981 |  0.2341 |  0.4784 |   0.1215 | 0.0761 |  0.1144 |     0.1444
      
      Change-Id: Ibbe17226dd47980da632814422d6201c9fc6fa36
      57f6bfd0
  2. 16 Nov, 2017 1 commit
    • Monty Montgomery's avatar
      Eliminate tx_size dependant shifts for Daala TX · a26262c3
      Monty Montgomery authored
      short-circuit av1_get_tx_scale to always return zero when
      CONFIG_DAALA_TX, and remove it from the actual Daala TX toplevel
      
      This has potential overflow consequences for any metrics computation
      based on pixels; as such, also force use of the high-bitdepth path in
      each of these case.
      
      subset-1:
      monty-rest-of-stack-baseline-s1@2017-11-13T00:39:03.881Z ->
      monty-rest-of-stack-noshift-s1@2017-11-13T14:37:42.541Z
      
         PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      -0.0030 | -0.0523 |  0.2656 |  -0.0239 | -0.0033 | -0.0029 |     0.0067
      
      objective-1-fast --limit=4:
      monty-rest-of-stack-baseline-o1f4@2017-11-13T00:37:06.999Z ->
      monty-rest-of-stack-noshift-o1f4@2017-11-13T14:37:16.992Z
      
         PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      -0.0264 |  0.2303 |  0.0822 |  -0.0109 | -0.0395 | -0.0709 |     0.0538
      
      Change-Id: I57da71861f105dc7a404fa75a75bde573855ef79
      a26262c3
  3. 12 Nov, 2017 1 commit
    • Monty Montgomery's avatar
      Simplify Daala forward TX toplevel for constant shift · a2d40a39
      Monty Montgomery authored
      Rather than backing out all the LGT-related shifting matrices
      throughout the existing TX code, separate out and simplify Daala
      forward TX into a single dedicated entry point.  When DAALA_TX is
      enabled, CONFIG_HIGHBITDEPTH is also forced, and all of Daala TX
      (lowbd and highbd) uses this single TX dispatch.
      
      At present, this should result in no effective functional change,
      however rectangular transforms are now always column-first-- that
      has minor rounding effects.
      
      subset 1:
      monty-daalaTX-fulltest-DaalaRDO-s1@2017-11-07T00:02:56.282Z ->
       monty-daalaTX-fulltest-fwd-s1@2017-11-07T03:08:55.478Z
      
         PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      -0.0576 |     N/A | -0.2646 |  -0.0125 | -0.0439 | -0.0479 |    -0.1798
      
      objective 1 fast:
      monty-daalaTX-fulltest-DaalaRDO-o1f4@2017-11-07T05:59:50.180Z ->
       monty-daalaTX-fulltest-fwd-o1f4@2017-11-07T06:00:08.500Z
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      0.0036 |  0.0477 |  0.1132 |   0.0863 | -0.0017 |  0.0209 |     0.0240
      
      Change-Id: I182a5c4388c410cbea8810e2f9e36fd37a4a46e5
      a2d40a39