• David Barker's avatar
    Miscellaneous fixes for var-tx · 16c64e33
    David Barker authored
    Lots of small bug fixes, mainly around the transform size coding:
    
    * The loop filter was accidentally using the non-subsampled
      block size for the V plane, due to comparing a plane index
      (0, 1, or 2) against PLANE_TYPE_UV (== 1)
    
    * We allowed an initial update of the transform partition probabilities
      even on frames where we know they will never be used
      (because tx_mode != TX_MODE_SELECT).
      Further, these probabilities would not be reverted at the end
      of the frame, leading to the probability delta persisting across frames.
    
      Change this to behave more like the non-var-tx transform size coding,
      where probability deltas are only coded for frames with
      tx_mode == TX_MODE_SELECT, and the deltas only apply for one frame.
    
    * Fix decoder for the case where the video as a whole isn't lossless,
      and we have tx_mode == TX_MODE_SELECT, but the current segment
      *is* lossless.
      Note that the encoder already does the right thing in this case.
    
    * Don't allow the transform splitting to recurse "below" 4x4.
      This is really just a refactor, but means we can increase the
      maximum depth when subdividing rectangular transforms if we
      want to, whereas the previous code would have needed special cases
      for 4x8 and 8x4 transforms.
    
    * Finally, when we hit the maximum splitting depth, don't update
      the counts as if we had coded a 'no split' symbol.
    
    Change-Id: Iaebdacc9de81d2e93d3c49241e719bbc02e32682
    16c64e33
Name
Last commit
Last update
aom Loading commit data...
aom_dsp Loading commit data...
aom_mem Loading commit data...
aom_ports Loading commit data...
aom_scale Loading commit data...
aom_util Loading commit data...
av1 Loading commit data...
build Loading commit data...
examples Loading commit data...
test Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.clang-format Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
.mailmap Loading commit data...
AUTHORS Loading commit data...
CHANGELOG Loading commit data...
CMakeLists.txt Loading commit data...
LICENSE Loading commit data...
PATENTS Loading commit data...
README Loading commit data...
README.md Loading commit data...
aomdec.c Loading commit data...
aomenc.c Loading commit data...
aomenc.h Loading commit data...
aomstats.c Loading commit data...
aomstats.h Loading commit data...
args.c Loading commit data...
args.h Loading commit data...
codereview.settings Loading commit data...
configure Loading commit data...
docs.cmake Loading commit data...
docs.mk Loading commit data...
examples.mk Loading commit data...
ivfdec.c Loading commit data...
ivfdec.h Loading commit data...
ivfenc.c Loading commit data...
ivfenc.h Loading commit data...
keywords.dox Loading commit data...
libs.doxy_template Loading commit data...
libs.mk Loading commit data...
mainpage.dox Loading commit data...
md5_utils.c Loading commit data...
md5_utils.h Loading commit data...
rate_hist.c Loading commit data...
rate_hist.h Loading commit data...
solution.mk Loading commit data...
tools.mk Loading commit data...
tools_common.c Loading commit data...
tools_common.h Loading commit data...
usage.dox Loading commit data...
usage_cx.dox Loading commit data...
usage_dx.dox Loading commit data...
video_common.h Loading commit data...
video_reader.c Loading commit data...
video_reader.h Loading commit data...
video_writer.c Loading commit data...
video_writer.h Loading commit data...
warnings.c Loading commit data...
warnings.h Loading commit data...
webmdec.cc Loading commit data...
webmdec.h Loading commit data...
webmenc.cc Loading commit data...
webmenc.h Loading commit data...
y4menc.c Loading commit data...
y4menc.h Loading commit data...
y4minput.c Loading commit data...
y4minput.h Loading commit data...