1. 13 Sep, 2017 5 commits
    • Rupert Swarbrick's avatar
      Simplify get_partition() implementation · 136d5c17
      Rupert Swarbrick authored
      This function is given a bsize and an mi array and has to figure out
      what partition to use to divide the given bsize in the direction of
      the sizes it finds in the mi array. (Since each block size can be
      reached by only one sequence of partitions, this can be done
      The previous version was correct, working by looking up entries in the
      partition_lookup array. Unfortunately, that lookup isn't quite enough
      when CONFIG_EXT_PARTITION_TYPES is true, so it then had to do some
      slightly confusing computations to fix things up after the fact.
      The new version should be more self-explanatory and doesn't work by
      looking things up in a magic array. It looks up the width and height
      corresponding to bsize and compares them with the width and height
      corresponding to the sb_type at mi_row,mi_col in the mi array. When
      CONFIG_EXT_PARTITION_TYPES is false, this is all you need, and the
      four corresponding cases can be found by a lookup in an array of 4
      With extended partition types and a sufficiently large block, you have
      to do a bit more searching. For example, if bsize is BLOCK_16X16 and
      the subsize is BLOCK_8X8, the partition might be PARTITION_SPLIT, but
      it might be one of PARTITION_HORZ_A or PARTITION_VERT_A instead. The
      new code adds some comments to explain what's going on.
      A nice side-effect of rewriting get_partition in this way is that it
      lets us completely dispense with the partition_lookup array.
      The patch also fixes comments for the A/B extended partitions in
      enums.h, which were slightly backwards (a "horizontal split means two
      blocks vertically above one another)
      Change-Id: I4b48189103aa63e1859f25a15d7690d53ca7baf5
    • David Barker's avatar
      aom-qm: Fix overflow in quantizer calculation · 568b7c7e
      David Barker authored
      By using a combination of a small qmlevel and large qindex,
      it is possible for the quantizer 'dqv' to end up > 2^15.
      This currently causes dqv to overflow, so widen its type
      to avoid that.
      Note that this is certainly an extreme case, but not entirely
      useless - if using a 32x32 transform, the dequantized value
      can be as small as dqv/2, which may still fit in 16 bits.
      Change-Id: I0451e70ba6b17620d94386a98d873f0a8c615635
    • Rupert Swarbrick's avatar
      Report a proper error when bitstream wrongly requests prev_frame_mvs · bdfb4a6f
      Rupert Swarbrick authored
      Using an assert (which was compiled out) meant we got to a
      use-after-free in bug 729.
      Change-Id: I4d0fa5e80322c395f94565c76e2285782c328505
    • Debargha Mukherjee's avatar
      Change/refactor compound mode handling for sub8x8 · 0f248c46
      Debargha Mukherjee authored
      Turn off compound modes as long as one of the dimensions
      is less than 8.
      Imapct on AWCY (0.05% increase in BDRATE)
      Change-Id: I4a70890c04149246a50e60990dede21cb8052fad
    • Yushin Cho's avatar
      Fix compile error when both of dist_8x8 and pvq are enabled · 39086c0b
      Yushin Cho authored
      Remove mistakenly leftover code from commit
      Note that this change force-disables activity masking in PVQ until we can add some
      bitstream signaling for it.
      Change-Id: I5279f82e7977733ebc40f5aed844ecec1279992a
  2. 12 Sep, 2017 7 commits
    • Zoe Liu's avatar
      Remove duplicate line of code · 10a051fc
      Zoe Liu authored
      Change-Id: Ie92a17d6c530bdd65464fbc599ce92561b8c0428
    • Jingning Han's avatar
      Remove unnecessary rate cost setup in lv-map encoding · ba42e749
      Jingning Han authored
      This won't change the coding statistics.
      Change-Id: Ibe3790836c59462f830029528d88a6e615efa493
    • Debargha Mukherjee's avatar
      Ensure consistent border extension before LR · fc9f3ccd
      Debargha Mukherjee authored
      Before LR the border extension needs to be consistent.
      Otherwise there will be inconsistency at the frame boundaries
      causing mismatches.
      Change-Id: I5b0c51aa1493c58f7acc706bf686ce711eadd75d
    • Nathan Egge's avatar
      Revert "Add an orthonormal 4-point Type-VII DST." · 5a5e1adb
      Nathan Egge authored
      This reverts commit 72c99e1a.
      No metrics on this commit.
      Change-Id: I9fc350b25e710c3d5d6d8299ab5348e8b31b39ea
    • Nathan E. Egge's avatar
      Add an orthonormal 4-point Type-VII DST. · 72c99e1a
      Nathan E. Egge authored
      Replaces the orthonormal Type-IV DST with an orthonormal Type-VII DST
       in od_bin_fdst4() and od_bin_idst4()
      Change-Id: I4ff0888e740d8cc063a2e5deaeceef7cb0d80485
    • David Barker's avatar
      Clarify comment in av1_set_mb_mi() · ee0fd929
      David Barker authored
      The code currently pads the decoded frame width and height to a
      multiple of 8 luma pixels, but there is a TODO suggesting that
      this may be changed to only require a multiple of 4 in future.
      But, as per the comments on the linked bug, there are good reasons
      to keep the decoded width and height as multiples of 8. So delete
      the outdated TODO and instead outline the reasons why the current
      behaviour is helpful.
      Change-Id: I2340bbcd740afe74c2e6fb3cf2e7a420db2b4f40
    • Luc Trudeau's avatar
      Chroma Fix for Superblock CDF update · d7150530
      Luc Trudeau authored
      UV_INTRA_MODES must be used for chroma modes
      Change-Id: I3e72d16204d11062a850fd00fb6272358dd61aac
  3. 11 Sep, 2017 9 commits
    • Sarah Parker's avatar
      Tokenize and write mrc mask · 99e7daa2
      Sarah Parker authored
      This allows a mask for mrc-tx to be sent in the bitstream for
      inter or intra 32x32 transform blocks. The option to send the mask
      vs build it from the prediction signal is currently controlled with
      a macro. In the future, it is likely the macro will be removed and it
      will be possible for a block to select either method. The mask building
      functions are still placeholders and will be filled in in a followup.
      Change-Id: Ie27643ff172cc2b1a9b389fd503fe6bf7c9e21e3
    • Jingning Han's avatar
      Use per tile model update for coeff_br coding · e29c9260
      Jingning Han authored
      Change-Id: Ie52d52bc25e3fdfdea877349215431d8edc064a3
    • Jingning Han's avatar
      Fix base range counter index in lv-map decoding · 2e40574e
      Jingning Han authored
      Fix the index order of br_idx.
      Change-Id: I55acda816d1ec33c25ca4c31b09976f5e6f9c562
    • Soo-Chul Han's avatar
      add SEG_LVL_ZEROMV · a752d1d5
      Soo-Chul Han authored
      Change-Id: Icd04302886a4d12890d04f9f15563169a91e3a0d
    • Zoe Liu's avatar
      Add support of multi-layer GF group 16 to bit allocation · c3903ff2
      Zoe Liu authored
      Change-Id: I52a7ae7404f15f29a0a0b892884cf0edea619507
    • Zoe Liu's avatar
      Define coding parameters for GF group of 16 · 8679f417
      Zoe Liu authored
      Propose the coding structure for GF Group of 16 to use the following
      multi-layer design:
        1st Layer: Frame 0 and Frame 16 (ALTREF)
        2nd Layer: Frame 8 (ALTREF2)
        3rd Layer: Frame 4 and 12 (ALTREF2)
        4th Layer: Frame 2, 6, 10, and 14 (BWDREF)
        5th Layer: Frame 1, 3, 5, 7, 9, 11, 13, and 15
      This CL implements the reference frame update for a GF group of 16
      using a look-up table, which is different from the current
      implementation. The motivation behind is to try out various GF group
      coding structures simply through manipulating the look-up table,
      instead of making changes on swapping the reference indexes and
      associated flags throughout the codebase.
      Change-Id: I823edae1c26263e0e0642a580685a25e81caa0a9
    • Debargha Mukherjee's avatar
      Change vertical border to 1 (2-line buffer) · da46da8e
      Debargha Mukherjee authored
      Changes the macros SGRPROJ_BORDER_VERT and
      SGRPROJ_BORDER_VERT to 1 to use a single line border
      vertically - which will be equivalent to a 2-line buffer.
      Change-Id: I788d9dca53d3d492058914215acf61e9d3d3880d
    • Steinar Midtskogen's avatar
      CDEF: Do not filter chroma if subsampling_x != subsampling_y · 1c1161f1
      Steinar Midtskogen authored
      Since CDEF looks uses the luma direction for chroma, CDEF would have
      to change significantly to support formats like 4:2:2.  The limited
      use of such formats does not justify the complexity to support this,
      so the simple solution is to mandate that the chroma planes aren't
      filtered if subsampling_x != subsampling_y.  Most of the visual gain
      is in luma, anyway.
      This also means that the chroma strengths and chroma skip condition
      shall not be sent if subsampling_x != subsampling_y.
      Change-Id: I35c184a6fe0908ae0fee1e74494b6904fa9a3c82
    • David Barker's avatar
      cdef-singlepass: Fix integer overflow · 0123bc7d
      David Barker authored
      When cdef-singlepass is enabled, it is possible to signal the
      CDEF parameters in such a way that we end up with unsigned
      integer overflow in constrain().
      Fix this by i) using signed instead of unsigned values,
      and ii) clamping the result to avoid going on to shift
      by a negative amount.
      Change-Id: Ib677b2d644e44000c54959f7280e646bf02054da
  4. 10 Sep, 2017 5 commits
    • Jingning Han's avatar
      Rework base range entropy coding in level map system · 87b01b5a
      Jingning Han authored
      Replace the truncated geometric distribution model with the grouped
      leaves structure for more efficient probability modeling.
      Each group has its own Geometric distribution
      This give us 0.2% gain on lowres
      Change-Id: If5c73dd429bd5183a8aa81042f8f56937b1d8a6a
    • Angie Chiang's avatar
      Re train nz_map prob · 698a6185
      Angie Chiang authored
      We got 0.07% gain on lowres
      Change-Id: I0aef14d16025d9933ec3d3b71086f3f55c81df66
    • Debargha Mukherjee's avatar
      Refactoring/simplification of buffers used for sgr · 1330dfd1
      Debargha Mukherjee authored
      Inlcudes miscellaneous cleanups, test fixes, and code reorganization
      for loop-restoration components.
      Change-Id: I5b2e6419234d945e6f4344b22636119b50df4054
    • Debargha Mukherjee's avatar
      Reduce/Eliminate line buffer for loop-restoration. · e168a783
      Debargha Mukherjee authored
      This patch forces the vertical filtering for the top and bottom
      rows of a processing unit for the Wiener filter to not use border
      more than what is set in the WIENER_BORDER_VERT macro.
      This macro is currently set at 0 to eliminate line buffer completely,
      but it could be increased to 1 or 2 to use limited line buffers
      if the coding efficiency is affected too much with a 0 line-buffer.
      Also, for the sgr filter we added the option of using overlapping
      windows horizonttally and vertically to improve coding efficiency.
      The vertical border used is set by the SGRPROJ_BORDER_VERT
      macro, while the horizontal border can be set by the
      SGRPROJ_BORDER_HORZ macro set at 2, the max needed. Currently we do not
      recommend changing SGRPROJ_BORDER_HORZ below 2.
      The overall line buffer requirement for LR is twice the max of
      Currently both are set as 0, eliminating line buffers completely.
      Also this patch extends borders consistently before CDEF / LR.
      Change-Id: Ie58a98c784a0db547627b9cfcf55f018c30e8e79
    • Luc Trudeau's avatar
      Remove useless (void)s · db4ccd42
      Luc Trudeau authored
      Change-Id: I70a4dd8095276eb53050eacca2ee33ba39e7cc74
  5. 09 Sep, 2017 5 commits
  6. 08 Sep, 2017 9 commits