1. 25 Oct, 2017 1 commit
    • David Michael Barr's avatar
      [CFL] Switch to txfm_rd_in_plane in alpha search · 1f8d0950
      David Michael Barr authored
      This is more precise than the dist functions it replaces.
      
      Results on Subset1 (compared with previous commit with CfL enabled)
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0634 | -0.9188 | -0.9429 |   0.0609 | 0.0722 |  0.0593 |    -0.3226
      
      Change-Id: I955a7d7eceea50482edb40b0d1041b300e3c9042
      1f8d0950
  2. 24 Oct, 2017 3 commits
  3. 23 Oct, 2017 1 commit
    • Yaowu Xu's avatar
      Use type with smaller size · abe5215f
      Yaowu Xu authored
      This is to reduce the size of the huge token buffer by at least more
      than 10%.
      
      BUG=aomedia:940
      
      Change-Id: I2ee7f7f62e8e8fb819cf7fba40a7ca7f860acaa4
      abe5215f
  4. 21 Oct, 2017 2 commits
  5. 20 Oct, 2017 3 commits
  6. 19 Oct, 2017 2 commits
    • Yue Chen's avatar
      Disable residue hash feature on cross-border blocks · 25dc0701
      Yue Chen authored
      Disable this feature unless the entire block is within the frame.
      The reason is, rd decisions in mbmi, e.g. inter_tx_block[][], made
      for blocks partially out of the border can be partly nonsense
      therefore cannot be reused by blocks at other locations.
      
      It caused an infinite loop when encoding a clip with repetitive
      patterns. A cross-border block has an invalid big tx stored
      in inter_tx_block[0][1] and the other block (same residue, within
      frame) reused this mbmi, which makes encoder never reach the
      termination condition when tx blocks are being recursively
      partitioned.
      
      BUG=aomedia:913
      
      Change-Id: Id25a1dbc4a68b5136f6bdf9f6b5811b7ec6920b0
      25dc0701
    • Luc Trudeau's avatar
      [CFL] Fix negative rounding issue in alpha dist · a45b104a
      Luc Trudeau authored
      get_scaled_luma_q0(-alpha_q3, pred_buf_q3[i]) is NOT equivalent to
      -get_scaled_luma_q0(alpha_q3, pred_buf_q3[i]). When the product
      alpha_q3*pred_buf_q3[i] is an exact multiple of 32 (0.5 in Q6), then the right
      shift will round both positive and negative values towards infinity, creating a
      bias. So, e.g., get_scaled_luma_q0(-4, 8) will yield 0, but
      -get_scaled_luma_q0(4, 8) will yield -1.
      
      Results on Subset1 (compared to parent With CfL enabled)
        PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000
      
      https://arewecompressedyet.com/?job=cfl-no-round-fix%402017-10-07T11%3A50%3A47.711Z&job=cfl-round-fix%402017-10-07T02%3A15%3A51.359Z
      
      Change-Id: I8a7900c32fbd7213f1ed4e09c3626c063800e186
      a45b104a
  7. 18 Oct, 2017 4 commits
  8. 17 Oct, 2017 7 commits
    • Yue Chen's avatar
      new_multisymbol: use cdf-based costs of interintra · eaf128a9
      Yue Chen authored
      Calculate rate costs of interintra and wedge interintar symbols
      based on cdfs that are per-sb updated.
      
      Change-Id: I4fa73953cb0bccbf078d2e2dd4e90370972fcce1
      eaf128a9
    • Yue Chen's avatar
      new_multisymbol: use cdf-based costs of palette flags · dab2ca9d
      Yue Chen authored
      The modification is only applicable to palette_y_mode and
      palette_uv_mode. Welcome to make changes to other palette syntax.
      
      Change-Id: I7bf0a49c06a3986475076fe291e26f4b783b8ab9
      dab2ca9d
    • Yushin Cho's avatar
      [dist-8x8] Add more asserts · 1cd34621
      Yushin Cho authored
      Added more asserts for dist-8x8 running in PSNR mode,
      i.e. with the encoder option "--enable-dist-8x8=1"
      instead of --tune=[cdef-dist | daala-dist].
      
      The asserts checks whether a 8x8 distortion measured on reconstructed 8x8 pixels
      is identical to the sum of distortions from sub8x8 partitions (or tx blocks for
      var-tx case).
      
      Change-Id: I14f2b24e674a9cbbe60e663449fc4e7f46f0e481
      1cd34621
    • Alexander Bokov's avatar
      Improving the model for pruning the TX type search · 0c7eb10d
      Alexander Bokov authored
      Introduces two new TX type pruning modes that provide better
      speed-quality trade-off compared to the existing ones. A shallow
      neural network with one hidden layer trained separately for each
      block size is used as a prediction model. The new modes differ in
      thresholds applied to the output of the neural net, so that they
      prune different number of TX types on average.
      
      Owing to relatively low quality loss PRUNE_2D_ACCURATE is used
      by default, regardless of speed settings. Starting with speed
      setting of 3 we switch to PRUNE_2D_FAST mode to get better
      speed-up.
      
      Evaluation results:
      ----------------------------------------------------------
      Prune mode | Avg. speed-up | Quality loss | Quality loss
                 |(high bitrates)|   (lowres)   |   (midres)
      ----------------------------------------------------------
      PRUNE_ONE  |     18.7%     |    0.396%    |    0.308%
      ----------------------------------------------------------
      PRUNE_TWO  |     27.2%     |    0.439%    |    0.389%
      ----------------------------------------------------------
      PRUNE_2D_  |     18.8%     |    0.032%    |    0.063%
      ACCURATE   |               |              |
      ----------------------------------------------------------
      PRUNE_2D_  |     33.3%     |    0.504%    |     ---
      FAST       |               |              |
      
      Change-Id: Ibd59f52eef493a499e529d824edad267daa65f9d
      0c7eb10d
    • Debargha Mukherjee's avatar
      Fix a compile bug with ext-partition-types · 0b34a79f
      Debargha Mukherjee authored
      Removes some stray CONFIG_CB4X4 config macros.
      
      BUG=aomedia:921
      
      Change-Id: Icc65e0b000f659d7fb18178c928a7bff7879f58c
      0b34a79f
    • Sebastien Alaiwan's avatar
      Remove abandonned CHROMA_2X2 experiment · d8b93f56
      Sebastien Alaiwan authored
      Change-Id: I5bff0a68602a89ce480fec049c8b2c4bce44f6bb
      d8b93f56
    • Hui Su's avatar
      intrabc: support var-tx · 12546aa2
      Hui Su authored
      Support recursive tx block partition.
      
      On the screen content testset, 0.2% gain for keyframe encoding.
      
      Change-Id: I623e6fbb910fef9c91617e02edf420019f67d189
      12546aa2
  9. 16 Oct, 2017 5 commits
  10. 14 Oct, 2017 1 commit
  11. 13 Oct, 2017 1 commit
  12. 12 Oct, 2017 3 commits
    • Yunqing Wang's avatar
      Find warped reference MV · 97d6a37e
      Yunqing Wang authored
      While finding the reference MV for a block, if one neighbouring block's
      motion mode is warped motion mode, instead of directly adding that
      block's MV to the candidate MV list, we use that neighbouring block's
      warped motion parameters to compute a MV for the center point of the
      current block, and then add that MV to the candidate MV list.
      
      Borg test result:
                   avg_psnr ovr_psnr ssim
      cam_lowres:  -0.507   -0.514  -0.685
      lowres:      -0.114   -0.122  -0.180
      
      The change is added under ext_warped_motion config flag.
      
      Change-Id: I3ce6290a1fd512b613eab5d7620c8bcb08f189a6
      97d6a37e
    • Yue Chen's avatar
      filter_intra: make fi mode index entropy coded · 63ce36fc
      Yue Chen authored
      Make fi mode index entropy coded instead of using raw bits. Mode
      cost estimation in key-frame RDO is updated as well. Modification
      to inter frame RDO is not included in this patch.
      Also key-frame y mode cdf table is re-trained since fi modes are
      attached to DC_PRED
      
      Key frame BDRate:
      -0.399% lowres, -0.339% midres
      
      Change-Id: I9ccf478b0a2e48fb1870fe8451e45e2c858a5f63
      63ce36fc
    • David Barker's avatar
      Make SEG_LVL_{SKIP,ZEROMV} blocks be single-ref-only · d92f3560
      David Barker authored
      This patch modifies the interpretation of SEG_LVL_SKIP and
      SEG_LVL_ZEROMV slightly, to fix a decoder crash and to save bits
      in the intended use cases of these segment flags.
      
      Previously, blocks using either of these segment flags could
      signal reference frames just like any other block. But the mode
      was implicitly taken to be ZEROMV. This worked fine in VP9, but
      crashed for compound blocks in AV1 since those should use
      ZERO_ZEROMV instead.
      
      Now we make it so that SEG_LVL_SKIP and SEG_LVL_ZEROMV imply
      that the block is single-reference. The reference to use is taken
      from the SEG_LVL_REF_FRAME segment feature if that is present,
      or is set to LAST_FRAME if not. See the attached bug report
      for the reasoning behind this.
      
      As a related change, we also teach the encoder how to deal with
      the combination of SEG_LVL_SKIP + SEG_LVL_REF_FRAME.
      
      BUG=aomedia:675
      
      Change-Id: I5e657cbfc1f08395a0301cba701edfb1682502a5
      d92f3560
  13. 11 Oct, 2017 2 commits
  14. 10 Oct, 2017 5 commits
    • Hui Su's avatar
      Add function to control palette usage · e87fb237
      Hui Su authored
      Add av1_allow_palette() to control whether palette mode should be enabled.
      
      Change-Id: Iee24636451be42eb36093dc3453bc39c7e686276
      e87fb237
    • Lester Lu's avatar
      lgt-from-pred: transforms based on prediction · 432012f6
      Lester Lu authored
      In this experiment, sharp image discontinuity in the predicted
      block is detected. Based on this discontinuity, we choose
      particular LGTs as row and column transforms.
      
      Bitstream syntax, entropy coding, and RD search for LGT are added.
      One binary symbol is used to signal whether LGT is used. This
      experiment can work independently with the lgt experiment.
      
      lowres: -0.414% for key frames, -0.151% overall
      midres: -0.413% for key frames, -0.161% overall
      
      Change-Id: Iaa2f2c2839c34ca4134fa55e77870dc3f1fa879f
      432012f6
    • Yushin Cho's avatar
      Use pixel domain skip error if possible in var-tx · 952eae29
      Yushin Cho authored
      When early skipped in var-tx, distortion is set the same as sse.
      If so, use pixel domain sse (i.e. skip error) since is more accureate
      than sse from transform domain.
      
      Change-Id: Id3cbc66ea6318108c031413646f3d06250e75e7e
      952eae29
    • Yushin Cho's avatar
      Fix that sse is added twice during early skip in var-tx · 16efec40
      Yushin Cho authored
      The rd_stats->sse is already updated by
      "rd_stats->sse += tmp << 4;",
      which is measured by pixel_diff_dist(), i.e. in pixel domain and
      w/o quantization().
      
      Change-Id: I4dc20a7e80af9dd846aa5de4298cb56e7f0d8f7e
      16efec40
    • Rupert Swarbrick's avatar
      Don't trash memory in select_tx_type_yrd · de2ea94e
      Rupert Swarbrick authored
      This patch fixes a bug in select_tx_type_yrd. The function works by
      looping over possible transform types to find the best option (calling
      select_tx_size_fix_type for each). Whenever there's a new best
      candidate, the code copies information about the transform from the
      mbmi structure into stack-allocated "best candidate" structures. At
      the end, it copies the "best candidate" data back to mbmi.
      
      Before the patch, if ref_best_rd was small, each call to
      select_tx_size_fix_type might return INT64_MAX (because they don't
      find anything better than ref_best_rd) and so we'd never actually copy
      anything to the "best candidate" structures. Then, at the end of the
      function, we'd merrily overwrite mbmi with whatever happened to be on
      the stack, causing general mayhem when something tried to read the
      data from mbmi later.
      
      This patch exits early if no candidates were found. It also adds an
      assertion saying that if no candidates were found, ref_best_rd must
      have been less than INT64_MAX. This should hopefully catch any bugs
      where the continue keywords in the loop stop us ever actually calling
      select_tx_size_fix_type.
      
      Change-Id: I54b998148281dd80f98d1570f736964593dc753f
      de2ea94e