1. 01 Nov, 2017 4 commits
  2. 31 Oct, 2017 2 commits
    • Luc Trudeau's avatar
      [CFL] 4:2:2 High Bit Depth · c8323c02
      Luc Trudeau authored
      Change-Id: I9f752dedfba29de9a4cfdd285c4b6dc32bd1630d
      c8323c02
    • Luc Trudeau's avatar
      [CFL] Add 4:2:2 Support · 0cfffa7f
      Luc Trudeau authored
      When 4:2:2 subsampling is used, both horizontal pixels are added and
      shifted left by 1 (instead of a right shift of 2 followed by a left
      shift of 3).
      
      Change-Id: Ib84c51cafabe0bd0de02dfe7868278e44d76f6db
      0cfffa7f
  3. 30 Oct, 2017 1 commit
    • Luc Trudeau's avatar
      [CFL] Sub8x8 Validation Code Rewrite · c7af36d4
      Luc Trudeau authored
      Sub8x8 Validation code is changed to be more robust. The scope of the
      validation is narrowed to validating that all of the required content in
      the storage buffer was stored between CfL predictions. The early
      termination used in the current mode decision code does not allow to
      validate more than that.
      
      This change does not change encoder output
      
      BUG=aomedia:925
      
      Change-Id: I7f1ed84da5037dcfaaf5da9cf33b4b8d664d2352
      c7af36d4
  4. 24 Oct, 2017 1 commit
  5. 07 Oct, 2017 3 commits
    • Luc Trudeau's avatar
      [CFL] Support for 4:4:4 High Bit Depth · 69d9e878
      Luc Trudeau authored
      Change-Id: I13ba0dbe57297b540b78512d21a119f05a86a849
      69d9e878
    • Luc Trudeau's avatar
      [CFL] Support for 4:2:0 High Bit Depth · 056d1f40
      Luc Trudeau authored
      high bit depth (_hbd) and low bit depth (_lbd) versions
      of the cfl functions: sum_above_row, sum_left_col,
      cfl_build_prediction, cfl_luma_subsampling_420 (4:4:4 will
      be added in subsequent commit) and cfl_alpha_dist. For
      cfl_alpha_dist, special care is given to scale the SSE
      according to the bit depth.
      
      BUG=aomedia:835
      
      Change-Id: I5b72845100d88fb8a438efe665bcae7fe1ba50b8
      056d1f40
    • Luc Trudeau's avatar
      [CFL] Extract sum top row and left col from custom DC_PRED · 13281d4f
      Luc Trudeau authored
      To simplify high bit depth commit, the summing the top row and the left
      column are extracted out of cfl_dc_pred. This does not change the
      bitstream.
      
      Change-Id: I5c9fe91df4942f736c5af29c1d93abb3a6c8501f
      13281d4f
  6. 29 Sep, 2017 1 commit
  7. 28 Sep, 2017 3 commits
  8. 27 Sep, 2017 4 commits
  9. 26 Sep, 2017 1 commit
  10. 19 Sep, 2017 3 commits
    • Luc Trudeau's avatar
      [CFL] Use MAX_NUM_TXB_SQUARE instead of MAX_NUM_TXB · f1cf7178
      Luc Trudeau authored
      MAX_NUM_TXB represents the maximum number of transform blocks in 1
      dimension. CfL requires the maximum number of transform blocks in 2
      dimensions. As such, we now use MAX_NUM_TXB_SQUARE instead of the
      erroneous MAX_NUM_TXB for the average buffer.
      
      There's an assert guarding for overrun on the average buffer.
      This fix stops assertions on 4:4:4 sequences. This patch does
      not alter AWCY results as AWCY only covers 4:2:0 sequences (for
      which MAX_NUM_TXB turns out to be sufficient). 
      
      Change-Id: I628db0131f60abc2d06cbbe3fe3dc40e28894ce3
      f1cf7178
    • Luc Trudeau's avatar
      [CFL] Replace Magic Numbers with Constants · 03b7e7d5
      Luc Trudeau authored
      Added CFL_SUB8X8_VAL_MI_SIZE and CFL_SUB8X8_VAL_MI_SQUARE to replace the
      magic numbers previously used. Also, CFL_SUB8X8_VAL_MI_SIZE goes from 2
      to 4 to support the upcoming EXT_PARTITION_TYPE experiment.
      
      Change-Id: I5c812af3366f80345a749451415660024e41ea3c
      03b7e7d5
    • Luc Trudeau's avatar
      [CFL] Refactor includes · 89ff793b
      Luc Trudeau authored
      The cfl_init function is moved out of cfl.h simplifying the includes and
      removing the need for forward declarations.
      
      Change-Id: I47312b25410b718a830b001391e386647005d57e
      89ff793b
  11. 31 Aug, 2017 1 commit
    • Luc Trudeau's avatar
      [CFL] Asserts for chroma_sub8x8 · c84c21c4
      Luc Trudeau authored
      When Chroma from Luma is combined with chroma_sub8x8, the prediction
      used for sub8x8 blocks originates from multiple luma blocks. Extra
      asserts are added to validate that the prediction buffer contains all
      the required information.
      
      Change-Id: I305c46ce9b8292697e1d5b181d123461026da11c
      c84c21c4
  12. 28 Aug, 2017 1 commit
    • Luc Trudeau's avatar
      [CFL] Move store flag to CFL_CTX · fcca37a4
      Luc Trudeau authored
      With recent changes, it is now possible to store the storage
      flag inside the CFL_CTX. This simplifies the implementation
      and will allow reuse in the decoder.
      
      This change does not alter the bitstream.
      
      Change-Id: Ibb8aebdd3d06f8765d40248ece8a038892e87032
      fcca37a4
  13. 08 Aug, 2017 1 commit
  14. 29 Jul, 2017 1 commit
    • David Michael Barr's avatar
      [CFL] Uniform Q3 alpha grid with extent [-2, 2] · f6eaa159
      David Michael Barr authored
      Expand the range of alpha to [-2, 2] in Q3.
      Jointly signal the signs, including zeros.
      Use the signs to give context for each quadrant
      and half-axis. The (0, 0) point is excluded.
      Symmetry in alpha_u == alpha_v yields 6 contexts.
      
      Results on Subset1 (Compared to 9136ab7d
      
       with CFL enabled)
      
         PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      -0.0792 | -0.7535 | -0.7574 |  -0.0639 | -0.0843 | -0.0665 |    -0.3324
      
      Change-Id: I250369692e92a91d9c8d174a203d441217d15063
      Signed-off-by: default avatarDavid Michael Barr <b@rr-dav.id.au>
      f6eaa159
  15. 25 Jul, 2017 1 commit
    • Luc Trudeau's avatar
      [CFL] Fix warnings when chroma_sub8x8 is disabled · 96b31516
      Luc Trudeau authored
      This change does not alter the bitstream
      
      Resuls on Subset1 (compared to 70a80a81 with cfl)
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000
      
      Change-Id: I7672eb4cde3c649ebba32610f7e56500e378c062
      96b31516
  16. 11 Jul, 2017 1 commit
    • Luc Trudeau's avatar
      [CFL] Convert cfl_alpha to q3 · 4e81d929
      Luc Trudeau authored
      Alpha's biggest fraction is 1/8, so Q3 does not change the bitstream.
      
      Results on Subset1 (compared to 503aca74 with CfL enabled)
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000
      
      Change-Id: I1fe5b2ace97179d5f950d7406a4f3d391924f89d
      4e81d929
  17. 10 Jul, 2017 1 commit
    • Luc Trudeau's avatar
      [CFL] Q0 DC_Pred · 7651b739
      Luc Trudeau authored
      The block level DC_PRED computed by CfL goes down from Q6 to Q0. This
      will allow to reuse existing assembly for DC_PRED and also reduce the
      requirements on the multilpy required to scale the reconstructed luma
      values
      
      Results on Subset1 (compared to f9684d222 with CfL enabled)
      
         PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      -0.0347 |  0.0229 | -0.1326 |  -0.0420 | -0.0057 | -0.0072 |    -0.0644
      
      Change-Id: I6ba82cc9e04fa4ab7c8ec40a7856deb273881748
      7651b739
  18. 06 Jul, 2017 5 commits
    • Luc Trudeau's avatar
      [CFL] Fewer bits for fixed point · 475fc9df
      Luc Trudeau authored
      Since alpha is Q3, we reduce y_average from Q10 to Q3. As such, the
      prediction is reduced from Q13 to Q6. Chroma dc_pred is reduced from Q7
      to Q6 in order to match with the prediction.
      
      Results on Subset1 (compared to 209de2e5b with CfL enabled)
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0010 |  0.0176 | -0.0538 |  -0.0043 | 0.0027 | -0.0097 |    -0.0018
      
      Change-Id: Ib7dd3968a764e0380ddc0ad2333ebacf1e9699cd
      475fc9df
    • Luc Trudeau's avatar
      [CFL] Convert dc_pred to fixed point · 2e6cb7e7
      Luc Trudeau authored
      The dc_pred values stored in the CfL context are in Q8.7 (Worst case
      division will be of 1/128).
      
      Results on Subset1 (compared to f9684d222 with CfL enabled)
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0118 | -0.0181 | -0.0109 |   0.0086 | 0.0086 |  0.0196 |     0.0018
      
      Change-Id: I0701e04fb76f03eff12ed01fd5fda675fbb15e32
      2e6cb7e7
    • Luc Trudeau's avatar
      [CFL] Fixed point implementation for tx average · bfe2827b
      Luc Trudeau authored
      This change does not impact the bitstream as no loss is incured by using
      a fixed point value for the transform size average.
      
      For low bit depth, the transform size average is stored using Q8.10
      fixed point format. Worst case, smallest fraction is 1/1024.
      
      Results on Subset1 (Compared to 366b74 with CfL)
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000
      
      Change-Id: Ia5b046b92a0e4c40e413b16af3394bdc0a8c8cd9
      bfe2827b
    • Luc Trudeau's avatar
      [CFL] Compute Average Over TX Block Instead of Pred Block · 03678940
      Luc Trudeau authored
      When computing alpha, multiple averages are computed, one for each
      transform block. The CfL prediction now uses the transform block average
      instead of partition block average.
      
      This allows the decoder to build the CfL prediction by using only the
      collocated reconstructed luma values for the current transform size and
      not the entire partition.
      
      Results on Subset 1 (Compared to 0e81b97c with CfL)
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0180 |  0.2627 |  0.2274 |   0.0233 | 0.0301 |  0.0312 |     0.1506
      
      A small regression is expected, this change was made to simplify
      hardware implementations.
      
      Change-Id: Ib2ce2a3053b85300c5c62ef0e3270af489568a38
      03678940
    • Luc Trudeau's avatar
      [CFL] clip CFL prediction to avoid overflow · 5c453db2
      Luc Trudeau authored
      The value predicted using CfL is clipped to avoid going out of the
      scope of the uint8. Both overflow and underflow was detected over
      Subtset1.
      
      Results on Subset1 (compared to 7e55571e with CfL enabled)
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0019 |  0.0001 |  0.0009 |   0.0047 | 0.0020 |  0.0023 |     0.0012
      
      Change-Id: Ie1190e2286aa90542eaa68b814cc5cfa031acb73
      5c453db2
  19. 03 Jul, 2017 1 commit
    • Luc Trudeau's avatar
      [CFL] Adjust Pixel Buffer for Chroma Sub8x8 · 780d249d
      Luc Trudeau authored
      Adjust row and col offset for sub8x8 blocks to allow the CfL prediction
      to use all available reconstructed luma pixels.
      
      Results on Subset 1 (Compared to b03c2f44 with CfL)
      
         PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      -0.1355 | -0.8517 | -0.4481 |  -0.0579 | -0.0237 | -0.0203 |    -0.2765
      
      Change-Id: Ia91f0a078f0ff4f28bb2d272b096f579e0d04dac
      780d249d
  20. 30 Jun, 2017 1 commit
  21. 29 Jun, 2017 2 commits
    • Luc Trudeau's avatar
      [CFL] Compute DC_PRED until end of frame · e7f9e16a
      Luc Trudeau authored
      When computing CFL's prediction block level DC_PRED,
      We can't assume that the neighbor will use the same
      transform size. As such, we don't know what pixels
      are available outside of the frame. A simple
      solution is to stop computing DC_PRED at the block
      boundary.
      
      Results on Subset 1 ( Compared to 9c6f8547 with CfL enabled)
         PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      -0.0015 | -0.0026 | -0.0025 |  -0.0034 | -0.0008 | -0.0044 |     0.0019
      
      Change-Id: I00ed4114e5c5871fdc7222fa2000474eb11d33e0
      e7f9e16a
    • Luc Trudeau's avatar
      [CFL] Better encapsulation · 3dc55e0f
      Luc Trudeau authored
      The function cfl_compute_parameters is added and contains the logic
      related to building the CfL context parameters. As such, many cfl
      functions can now be encapsulated inside of cfl.c and not exposed to the
      rest of AV1.
      
      This also allows for supplemental asserts that validate that the CfL
      context is properly built.
      
      Results on Subset1 (compared to 9c6f8547 with CfL)
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000
      
      Change-Id: I6d14a426416b3af5491bdc145db7281b5e988cae
      3dc55e0f
  22. 20 Jun, 2017 1 commit
    • Luc Trudeau's avatar
      [CFL] Fix stack smash in cfl_load · 5f3d370b
      Luc Trudeau authored
      The height padding offset was computed from the beginning of the pixel buffer
      and not the current position in the pixel buffer.
      
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
      0.0000 | -0.0001 |  0.0002 |  -0.0000 | -0.0000 | -0.0000 |    -0.0002
      
      Change-Id: Id20ec3a6395b11d0258ec1e100890fe4701b820c
      5f3d370b