1. 10 Oct, 2017 5 commits
    • Hui Su's avatar
      Add function to control palette usage · e87fb237
      Hui Su authored
      Add av1_allow_palette() to control whether palette mode should be enabled.
      
      Change-Id: Iee24636451be42eb36093dc3453bc39c7e686276
      e87fb237
    • Lester Lu's avatar
      lgt-from-pred: transforms based on prediction · 432012f6
      Lester Lu authored
      In this experiment, sharp image discontinuity in the predicted
      block is detected. Based on this discontinuity, we choose
      particular LGTs as row and column transforms.
      
      Bitstream syntax, entropy coding, and RD search for LGT are added.
      One binary symbol is used to signal whether LGT is used. This
      experiment can work independently with the lgt experiment.
      
      lowres: -0.414% for key frames, -0.151% overall
      midres: -0.413% for key frames, -0.161% overall
      
      Change-Id: Iaa2f2c2839c34ca4134fa55e77870dc3f1fa879f
      432012f6
    • Yushin Cho's avatar
      Use pixel domain skip error if possible in var-tx · 952eae29
      Yushin Cho authored
      When early skipped in var-tx, distortion is set the same as sse.
      If so, use pixel domain sse (i.e. skip error) since is more accureate
      than sse from transform domain.
      
      Change-Id: Id3cbc66ea6318108c031413646f3d06250e75e7e
      952eae29
    • Yushin Cho's avatar
      Fix that sse is added twice during early skip in var-tx · 16efec40
      Yushin Cho authored
      The rd_stats->sse is already updated by
      "rd_stats->sse += tmp << 4;",
      which is measured by pixel_diff_dist(), i.e. in pixel domain and
      w/o quantization().
      
      Change-Id: I4dc20a7e80af9dd846aa5de4298cb56e7f0d8f7e
      16efec40
    • Rupert Swarbrick's avatar
      Don't trash memory in select_tx_type_yrd · de2ea94e
      Rupert Swarbrick authored
      This patch fixes a bug in select_tx_type_yrd. The function works by
      looping over possible transform types to find the best option (calling
      select_tx_size_fix_type for each). Whenever there's a new best
      candidate, the code copies information about the transform from the
      mbmi structure into stack-allocated "best candidate" structures. At
      the end, it copies the "best candidate" data back to mbmi.
      
      Before the patch, if ref_best_rd was small, each call to
      select_tx_size_fix_type might return INT64_MAX (because they don't
      find anything better than ref_best_rd) and so we'd never actually copy
      anything to the "best candidate" structures. Then, at the end of the
      function, we'd merrily overwrite mbmi with whatever happened to be on
      the stack, causing general mayhem when something tried to read the
      data from mbmi later.
      
      This patch exits early if no candidates were found. It also adds an
      assertion saying that if no candidates were found, ref_best_rd must
      have been less than INT64_MAX. This should hopefully catch any bugs
      where the continue keywords in the loop stop us ever actually calling
      select_tx_size_fix_type.
      
      Change-Id: I54b998148281dd80f98d1570f736964593dc753f
      de2ea94e
  2. 09 Oct, 2017 4 commits
    • Sarah Parker's avatar
      Change rectangular vartx recursion depth to 2 · d25ef8c6
      Sarah Parker authored
      0.15% improvement on lowres set
      
      Change-Id: If16a8e07797c64508f9e2d9b26ae874ac53c57a4
      d25ef8c6
    • Urvang Joshi's avatar
      Revert wrong uses of TX_SIZE enum. · ab8840eb
      Urvang Joshi authored
      Introduced by: https://aomedia-review.googlesource.com/c/aom/+/25181
      
      Change-Id: I1f25178d6b273fbeade4c33f153b5f2bac4a8b99
      ab8840eb
    • Cheng Chen's avatar
      Match braces in VIM for rdopt.c · 1483a714
      Cheng Chen authored
      Change-Id: I23344af711d9a31b819fca35ae3ad3b7edf4852e
      1483a714
    • Rupert Swarbrick's avatar
      Define block_signals_txsize function · fcff0b25
      Rupert Swarbrick authored
      This returns true if a block signals tx_size in the stream and uses it
      in the bitstream writing code and the decoder.
      
      Note that we can't quite use it in pack_inter_mode_mvs when
      CONFIG_VAR_TX && !CONFIG_RECT_TX but I've switched the code to using
      it the rest of the time since rect-tx is adopted and eventually the
      other code path should be deleted.
      
      Also use the helper function in tx_size_cost in rdopt.c, where the
      test was wrong and caused underestimates of block
      costs. (Specifically, the code that subtracts tx_size_cost from
      this_rate_tokenonly in rd_pick_intra_sby_mode ended up subtracting
      zero for a 4x8 block).
      
      The behaviour of the decoder should be unchanged. The only change in
      the encoder's behaviour should be in tx_size_cost where it should now
      match the rest of the code.
      
      Change-Id: I97236c9ce444993afe01ac5c6f4a0bb9e5049217
      fcff0b25
  3. 08 Oct, 2017 1 commit
  4. 07 Oct, 2017 2 commits
    • Luc Trudeau's avatar
      [CFL] Support for 4:2:0 High Bit Depth · 056d1f40
      Luc Trudeau authored
      high bit depth (_hbd) and low bit depth (_lbd) versions
      of the cfl functions: sum_above_row, sum_left_col,
      cfl_build_prediction, cfl_luma_subsampling_420 (4:4:4 will
      be added in subsequent commit) and cfl_alpha_dist. For
      cfl_alpha_dist, special care is given to scale the SSE
      according to the bit depth.
      
      BUG=aomedia:835
      
      Change-Id: I5b72845100d88fb8a438efe665bcae7fe1ba50b8
      056d1f40
    • Debargha Mukherjee's avatar
      Remove the speed optimization for rd_stats_stack · 9245d89d
      Debargha Mukherjee authored
      This optimization for speed was useful only when max tx-size
      was 32x32. However with tx64x64 this was breaking certain assumptions
      causing huge drops in coding efficiency. So I am removing this
      optimization for now. This can be brought back latger as a speed feature.
      The removal of this optimzation brings back the loss when 32x64
      and 64x32 transforms are used.
      
      Change-Id: I15987ea9ff53fa36a2962fe5f156c30a11e809ed
      9245d89d
  5. 06 Oct, 2017 5 commits
    • Jingning Han's avatar
      Rework key frame intra mode context model · a45d842d
      Jingning Han authored
      Reduce the context model size for key frame modes from 30240 bits
      to 4500 bits, i.e., less than 1/6 of the original context model.
      The coding performance loss on key frame is 0.14% for lowres and
      noise level difference for video sequence. The loss on key frame
      for midres is 0.05% and noise level for whole video. The change
      on hdres kf coding is 0.015%.
      
      Change-Id: I9e36825e5c5ee6ba35038c3ca349ad1ad3429910
      a45d842d
    • Debargha Mukherjee's avatar
      Avoid large stack allocations · 5d108a36
      Debargha Mukherjee authored
      When ext-partition and ncobmc-adapt-weight is on, avoid too large
      stack allocations.
      
      Change-Id: I8db74e45cac80c4e5dfd9e20cfc73d9978d1578e
      5d108a36
    • Alexander Bokov's avatar
      Predict skip flag to speed up the TX type search · 8829a24d
      Alexander Bokov authored
      Average speed-up (lowres):
      low bitrates: 6.6%
      mid bitrates: 2.5%
      high bitrates: 0.0%
      
      Average PSNR loss:
      lowres: 0.010%
      midres: 0.005%
      
      Change-Id: Id34fb247e5e31f04ca324c58142e4b5ac4edacda
      8829a24d
    • Rupert Swarbrick's avatar
      Simplify the ALL_ZERO_FLAG logic in av1_rd_pick_intra_mode_sb · 799ff701
      Rupert Swarbrick authored
      Since the CONFIG_EXT_INTER #if/#endif lines have been removed, it's a
      bit clearer what's going on here and this patch cleans up the code.
      
      Firstly, the patch pulls the cheap checks on best_mbmode.ref_frame out
      to the front of the block, so we needn't call gm_get_motion_vector at
      all for compound predictions.
      
      Next, second element of the zeromv array is never used, so we needn't
      compute it.
      
      Finally, the patch removes the calls to lower_mv_precision. These
      shouldn't be needed, but it's not exactly obvious why not so the patch
      adds some comments to gm_get_motion_vector to explain what's going on
      and adds an assertion to make sure they are true. It also adds a call
      to integer_mv_precision on the early return path of
      gm_get_motion_vector, correcting an apparent bug when CONFIG_AMVR is
      true.
      
      This patch shouldn't make any difference to encoder or decoder
      behaviour.
      
      Change-Id: I0b4a01063574d080bbf6d30187f4e1748c60939d
      799ff701
    • RogerZhou's avatar
      Extend IntraBC to 4x4 · ca86546f
      RogerZhou authored
      Change-Id: I3f30c35bcd1bc623ad0c34c4b954ff71b2fcfd00
      ca86546f
  6. 05 Oct, 2017 2 commits
  7. 04 Oct, 2017 3 commits
    • Rupert Swarbrick's avatar
      Fix rate costing for small blocks with skip flag · c6cc1f5e
      Rupert Swarbrick authored
      In av1_rd_pick_intra_mode_sb, the code calculates the rate for Y and
      UV planes separately. If the transform coefficient should be zero,
      rd_pick_intra_sby_mode and rd_pick_intra_sbuv_mode return the cost of
      actually coding up the zero coefficient, but also set a flag (y_skip
      or uv_skip) saying that this could be skipped.
      
      Since the skip flag isn't per-plane, av1_rd_pick_intra_mode_sb checks
      to see whether both y_skip and uv_skip were true. In that case, it
      costs the block for setting the skip flag rather than outputting zero
      transform coefficients.
      
      If a small block (less than 8x8) has no chroma information,
      x->skip_chroma_rd is true. In that case, we don't call
      rd_pick_intra_sbuv_mode and so uv_skip is never set. However, when we
      come to write the block, it will be written using the skip flag. This
      patch gets the costing right in that case.
      
      Change-Id: Ib31b80b4b44a5c8ed9d9b3f86d782c54927345f3
      c6cc1f5e
    • Debargha Mukherjee's avatar
      Fix rd scales for transforms larger than 32x32 · b02d2f39
      Debargha Mukherjee authored
      Change-Id: I1ddec0cf3513e2bd7568393e5ed5d52c25014ab4
      b02d2f39
    • Rupert Swarbrick's avatar
      Pack InterpFilters into a single integer · 27e90295
      Rupert Swarbrick authored
      Before this patch, if CONFIG_DUAL_FILTER was true then an MB_MODE_INFO
      stored its filter choices as an array of four numbers, each of which
      was between 0 and 10. It also seems that elements 2 and 3 of the array
      were always the same as elements 0 and 1 when used.
      
      This patch defines a new type(def) called InterpFilters together with
      constructor and extractor functions. When CONFIG_DUAL_FILTER is zero,
      InterpFilters is a synonym for InterpFilter and the constructor and
      extractor functions should compile away to nothing. When it is
      nonzero, InterpFilters is a uint32_t which stores the x filter in the
      high part and the y filter in the low part (this looks strange, but
      matches the old numbering).
      
      Making this change allows us to get rid of lots of special case code
      that was dependent on CONFIG_DUAL_FILTER. The uniform
      extract/make/broadcast interface also actually shortens code in
      general.
      
      Change-Id: I6b24a61bac3e4b220d8d46d0b27cfe865dcfba81
      27e90295
  8. 03 Oct, 2017 1 commit
  9. 02 Oct, 2017 1 commit
  10. 30 Sep, 2017 2 commits
    • Yushin Cho's avatar
      [dist-8x8] Bug fix - exclude invisible pixels · 2f025aae
      Yushin Cho authored
      There were two places where invisible pixels are included
      when calculating distortion with dist-8x8 is turned on.
      Those are bugs and fixed by this commit.
      
      1) When av1_dist_8x8() is called in dist_8x8_sub8x8_txfm_rd()
      - New assert() are also added to detect any failures in the future.
      
      2) Inside av1_dist_8x8() and av1_dist_8x8_diff(), with tune-metric == PSNR.
      
      Change-Id: I078d1feb3bb010c0e34041640bb69a400125817a
      2f025aae
    • Yaowu Xu's avatar
      Remove experiment speed_refs · 9293bce0
      Yaowu Xu authored
      Change-Id: Ie4e2f04a6fa27480e500469a0d822e539b7cb066
      9293bce0
  11. 29 Sep, 2017 3 commits
  12. 28 Sep, 2017 2 commits
  13. 27 Sep, 2017 4 commits
    • Angie Chiang's avatar
      Only use fast optimize_b in rd loop for lv_map · 25645b74
      Angie Chiang authored
      Change-Id: Ia33e9962fce962e4f45281537d3e90b4d88543a5
      25645b74
    • Luc Trudeau's avatar
      Replace DC_PRED with UV_DC_PRED for chroma · 91357ee9
      Luc Trudeau authored
      When using uv_mode the corresponding enum is the UV_PREDICTION_MODE not
      the PREDICTION_MODE enum.
      
      Change-Id: I44b19e4db5947a2d44281430b34ed61b82f999eb
      91357ee9
    • David Barker's avatar
      ext-partition: Don't read not-yet-decoded values · 761b1ac8
      David Barker authored
      When deciding whether the top-right or bottom-left blocks are
      available, we currently always act as if we're using 128x128
      superblocks. This means that, when using 64x64 superblocks,
      we sometimes conclude that blocks are available when they haven't
      been decoded yet!
      
      This typically happens at, for example, mi_row=15, mi_col=16
      (for bottom left), where we're at a 64x64 boundary but not
      a 128x128 boundary.
      
      This patch fixes the issue by checking based on the signalled
      superblock size.
      
      Note: Most of this patch is just threading 'cm' through the
      intra prediction process, so that we have access to cm->sb_size
      in has_top_right() and has_bottom_left()
      
      Change-Id: I126964c510aafffc870e7cd8b3e64a46abb14b3a
      761b1ac8
    • Luc Trudeau's avatar
      [CFL] Subsample to Q3 · a0af3b5e
      Luc Trudeau authored
      Result from luma subsampling is left-shifted by 3. This avoids having to
      do it during averaging, in alpha search and when building the
      prediction. This change does not alter the bitstream.
      
      Results on Subset1
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000
      
      https://arewecompressedyet.com/?job=cfl-baseline%402017-09-06T17%3A41%3A38.041Z&job=cfl-SubsampleQ3%402017-09-06T17%3A42%3A01.252Z
      
      Change-Id: I6e89eac6496f7c36e46364c9223fbcbca6759032
      a0af3b5e
  14. 26 Sep, 2017 2 commits
  15. 25 Sep, 2017 1 commit
  16. 23 Sep, 2017 2 commits
    • Yushin Cho's avatar
      Improve dist-8x8 · 8e75e8bb
      Yushin Cho authored
      Improve dist-8x8 when computing 8x8 yuv dist for sub8x8.
      
      To apply dist-8x8 for sub8x8 partitions, once mode decision for
      sub8x8 partitions are finished then dist-8x8 is computed on 8x8 window.
      Since dist-8x8 is only for luma, chroma distortion should be identified.
      
      Previously, it has been hard to be free from potential bugs doing this,
      due to the complex inter mode search code.
      
      The new method is less-error-prone, which computes uv distortion (in MSE) after
      the mode decisions for all of sub8x8 blocks in a 8x8 window are finished,
      when the dist-8x8 distortion for luma 8x8 pixels are computed with
      new distortion metric.
      
      All the code separating y and uv distortion in inter mode search has
      been removed in this commit.
      
      Change-Id: Ieaccb7915df5faeb5e89a7e70b2b7cbac65231af
      8e75e8bb
    • Hui Su's avatar
      Move local functions out of blockd.h · 032ab8b3
      Hui Su authored
      Change-Id: Id667950de84ad6a0b55222264a0ce8473cd10bcc
      032ab8b3