Sub8x8 Validation code is changed to be more robust. The scope of the
validation is narrowed to validating that all of the required content in
the storage buffer was stored between CfL predictions. The early
termination used in the current mode decision code does not allow to
validate more than that.

This change does not change encoder output

BUG=aomedia:925

Change-Id: I7f1ed84da5037dcfaaf5da9cf33b4b8d664d2352

Backwards adaptation doesn't really make sense in this context,
because ext-tile means the decoder might not decode every tile in the
frame, in which case it can't work out how to adapt probabilities.

It turns out that the tests in the tree were passing when
SIMPLE_BWD_ADAPT was disabled because (presumably) all of them
happened to decode every tile, so backward adaptation worked as you'd
expect. With SIMPLE_BWD_ADAPT, there's some code in get_tile_buffers
to figure out which is the largest tile. This wasn't mirrored in
get_ls_tile_buffers, which caused the encoder and decoder to get out
of sync because they ended up with different CDFs. You also can't
easily fix things in get_ls_tile_buffers because that code is careful
not to calculate the size of tiles it doesn't need.

BUG=aomedia:917

Change-Id: Ia926692f86ca1466252108e09da3de590d45f048

Change-Id: I0cc53a03f07a11a6f7ea0570ff4ee8cf7c18c5aa

Remove the special case handling for the topmost/bottommost
rows in each processing unit. This causes slightly different
effects depending on whether striped-loop-restoration is enabled.

With striped-loop-restoration:
  Now that we explicitly fill out 3 rows of above/below pixels
  for each stripe, we don't need to use stepdown_wiener_kernel.
  Instead, the duplication of the topmost/bottommost pixels
  accomplishes the same task, while making the code much cleaner.

  This patch should not cause a change in output, except in a
  couple of cases which were already questionable. In particular,
  it fixes bug #953, where the Wiener filter could not handle
  small processing units (<4 rows high)

Without striped-loop-restoration:
  The Wiener filter returns to using a full 3 pixels above/below
  the processing unit. In order to make sure there are enough
  pixels, we need to expand WIENER_BORDER_VERT to 3 pixels.

  This will result in a slight change in output, but should be
  fairly minor.

BUG=aomedia:953

Change-Id: I9530ef55909246f7ba488b7ecfd92d59e776b2f9

Change-Id: I69d1c4b63960b46affb3f907fe8a1385f0de8948

Move #if CONFIG_EXT_TX in av1_fht32x32_test.c to be consistent as in
dct.c

BUG=aomedia:964

Change-Id: I0e7a777588f76a9088ce05ec8598f6ffa7c19e48

Replaces the lifting based orthonormal 4-point Type-IV DST with an
 orthonormal 4-point Type-VII DST that has no iterative multiplies.

Change-Id: I0a1f1a8d8cecce1c8002b7891baea601bc088690

Account for 1-D/2-D transform kernels for the eob modeling. To
maintain a smaller context cardinality, set the two 1-D transform
kernels in the same category. The difference in directions should
be largely covered by the scan order.

This and the previous CLs on nz_map context modeling together
improve the compression performance of level-map coefficient coding
system by 0.4% for lowres.

Change-Id: I8c4f03ca01ce3d248950d04bd1266f445b4227a0

Account for the rectangular transform block sizes in the non-zero
map context model.

Change-Id: I16cf21a4120c10c213df10950aeb4ef0ea40c477

Support compound_segment and wedge.
JNT_COMP is a competitor to compound_segment and wedge. They are
mutually exclusive.

Change-Id: Id93c36de44a69008182424f40325e42e699bedcd

For ext-intra direcation intra modes:

1. Use neighbor block modes to modify edge filtering strength
   Coding gain (lowres/midres/hdres):
     (8 keyframes)
     PSNR: -0.19 -0.22 -0.10
     SSIM: -0.29 -0.27 -0.13

2. Use context-based cdf to code angle_delta syntax
     (8 keyframes)
     PSNR: -0.20 -0.24 -0.27
     SSIM: -0.29 -0.33 -0.37

3. Filter corner sample:
     (8 keyframes)
     PSNR: -0.01 -0.02 -0.05
     SSIM: -0.03 -0.04 -0.05

Combined Bd-rate improvement for 8 keyframes
     PSNR: -0.40 -0.47 -0.40
     SSIM: -0.57 -0.60 -0.51

Change-Id: Id47ac17b6bf91cd810b70cacfc5b457341f417f3

Change-Id: Ia8916ac9ae2dc7f086057510cb239d1796f60801

Range is 9 to 16, and not 8 to 15.

BUG=aomedia:972

Change-Id: I7de6cea16a6377d9cd3b2af73efc841b42dad1fa

Change-Id: Ib4faac1e7da40a351ec3abfe1f636a94c92ef0a3

The compressed header size is now sometimes not signalled and the
encoder code hadn't been updated to match.

Note: This mistake caused the failing test tracked in bug 917 to fail,
      but fixing it doesn't fix the test: the decoder now staggers on
      a little further before getting out of sync.

BUG=aomedia:917

Change-Id: Ic0de99ede778844c4dbdd34dba3fba9c1fa2d95f

Since commit 6c545216, loop restoration coefficients are written inside
tiles data. Since the Wiener and Sgrproj coefficients are
delta-encoded (and tiles must be decoded independently) this means we
must call set_default_wiener and set_default_sgrproj at the start of
encoding and decoding each tile. That patch missed doing so when
encoding large scale tiles (part of the EXT_TILE experiment). This
adds the missing lines.

Note: This caused AV1/AVxEncoderThreadLSTest.EncoderResultTest/3 to
      fail with --enable-ext-tile, which is tracked in bug
      917. Unfortunately, there's something else going wrong that
      means the test is still failing: I'll address that in a separate
      patch.

BUG=aomedia:917

Change-Id: I5bd8e9d7f7a2abd60779abf170185a1c5b090474

Avoid resetting current_q_index.

Change-Id: I76d27605b6a8c0cd6b1032e24ea5ca8561115e5a

We reduce max stack size from 16 to 8.

Memory reduction:
- peak usage for 1080p video: 2.328 GB → 1.788 GB
- sizeof ref_mv_stack: 6144 → 3072
- sizeof(MB_MODE_INFO_EXT): 6456 → 3384
- sizeof(PICK_MODE_CONTEXT):8056 → 5000
- sizeof(PC_TREE): 201440 → 125040

Compression performance is roughly neutral:
- AWCY objective-1-fast: +0.03
- Google lowres: 0.0
- Google midres: -0.006

BUG=aomedia:940

Change-Id: Ifd38359c58e40b1c94552c5034618da8ce510f62

Clamp in range [-MAX_LOOP_FILTER, MAX_LOOP_FILTER], for experiment LOOPFILTER_LEVELS.

BUG=aomedia:970

Change-Id: Ica52242e969dcea49565f118155de5eef32a4cc4

Let jnt_comp work with compound_segment. When frame distance weights
are used, compound_segment blending is turned off. When normal compound
mode is used, compound_segment blending is searched. The rdcost should
select between {frame distance weights, normal compound,
compound_segment blending}.

Change-Id: I162d1d204ba6a5976538357d9dbc71fc5b1aa8d5

Change-Id: I0e6f7ab981e31f7120105515f6204568b6dc82d3

Select the best compound_idx in rd.
The rate/cost for compound_idx and their ctx will be in patch 4.

But there's a bug for now if we don't encode one more time using the
selected compound_idx. It remains a issue to be solved in the future.

Change-Id: I5e1ba51da2b6ab5bacd8aba752dda43bd2257014

The 13-tap versions of the wide filters are not ported to SIMD yet
Also add #if to hide a few unused static functions

Change-Id: I1accb2ebf3774a64529d08d667cbebd608d37d39

This (correctly) triggered a compiler warning. The patch is very
simple, and tries to match the equivalent logic in
loopfilter_frame (in encoder.c)

Change-Id: I62ca11dffe0f5c21f81c1db2e2dcafdb08cef83c

Reduce the motion interpolation filter taps for inter prediction
blocks with widths or heights smaller than or equal to 4 to alleviate the memory
bandwidth increase.

AWCY HL: 0.01% Y, -0.20% U, -0.29% V (positive number means loss)

Change-Id: Ic454340e20aea2f1aae622336990f24a9e5b54d8

It seems that the change in commit 28444be6 missed this line (which
only fails to compile with --enable-debug because it's in an assert)

BUG=aomedia:952

Change-Id: Ibe6200b2928d198c73f3e9c9f0badcbdaffcac49

Save and restore 3 rows above and below each stripe, instead of 2.
The extra rows are filled with duplicates of the outermost context
rows.

This should not affect the encoder or decoder output in any way,
as currently these outer rows are not used. But this will enable
later patches to simplify the code and make it a closer match
to the way things are described in the striped-loop-restoration
design document.

Change-Id: I8ae5433e321d6025c6dc1b473330f485f1599340

When needed, fallback regular interp filter at reconstruction stage.

Such bitstreams are valid.
However, as we don't expect aomenc to generate them,
print a helper warning.

Change-Id: If30c8d8e478688d142abd857f4c35f3e8c68edb4

This allows time to investiage the bug related to issue#917, adn the
experiment should be re-enabled after a proper fix is in place for
the issue.

BUG=aomedia:917

Change-Id: I12f65bbe9fa80e823d0949533c3df0c60790ffeb

Change-Id: I567420e45f54cfe991065614d0a8c0c4d637e116

Change-Id: Ibfeb424bf0ebab7bbeb69f6f6df24a4f4924ec97

This commits moves coding of signs in a transform block into its own
pass. Grouping signs coded in literal mode together may help improve
throughput.

Change-Id: I999c6eeeb780b188ab9e2c6ea888f9de4dbce893

Change-Id: I6d0f42ddec857921f5beb52b25925747e6412956

The last restoration unit in a tile is allowed to be up to 1.5x
the nominal restoration unit size. This was not properly accounted
for in the definition of RESTORATION_LINEBUFFER_WIDTH, leading to
memory corruption whenever we hit a particularly wide restoration
unit.

Change-Id: I6e858278bf1e3304eedb5f974f1db6961245e7bf

Change-Id: Ib36a8df1a3ebddbf4320fb7b9b5537041bddc3a3

Use br-node approach, which can be easily turned into multi-symbol
if desired.

Change-Id: I40df5178ab299af24d347d91f01a88dbfc9305a6

Change-Id: I2ae2a33574bc3072561e696a31e0ea2e0770afa9

Change-Id: Idcb0a6660ac3b34eb79c216d71c8a71ffb863669

Change-Id: Idb805e3be693ff9d945998dea99c22fa6284bc04

Change-Id: If77800c0904b5e004508274acb32ae46a641405b