Change-Id: Id721adcec7bd260f05690ee26890dd1180a74784

BUG=aomedia:818

Fixing "NIghtly test "AV1/ResizeCspTest.TestResizeCspWorks/0" fails"

The flag was accidentaly turned on in the commit
c00769a9.

Change-Id: I9d7102022fddedc9a3e35ae8086159f06067d21c

Change-Id: Ia58759dec0c6d5e5d0e91166e2abf5204c3217f3

Use them to simplify the body of compare_img in encode_test_driver.cc.

Change-Id: I4fbb4e67917f61f224e3d1b1a6970d10fc3cd777

Change-Id: I3209d0074bf3ed1fda260dc2f5cacbfa616ad6ae

We calculate eob cost in the end of the outer for loop
However, The continue statement from br_node may lead to ignoring
eob cost

Change-Id: Ic80c88e034836eb4dc4ac8b87bf0b7b45bf5d097

This is for reduce memory usage for adapt_scan
The whole change will be under the flage USE_2X2_PROB

Change-Id: If7839d6396dad7618155ef2f36896d17743696ce

Change-Id: I88e425c55b61629430078e11b5b94bac164d027e

Removes setup in compressed header so that the compressed
header can be skipped.

Change-Id: I8d7ed63944a4f75704222f4ad7d0159db1cec0d1

Change-Id: I3d6a440b13473b0d1e8b18c6629c7e8e3fb446ae

Improve dist-8x8 when computing 8x8 yuv dist for sub8x8.

To apply dist-8x8 for sub8x8 partitions, once mode decision for
sub8x8 partitions are finished then dist-8x8 is computed on 8x8 window.
Since dist-8x8 is only for luma, chroma distortion should be identified.

Previously, it has been hard to be free from potential bugs doing this,
due to the complex inter mode search code.

The new method is less-error-prone, which computes uv distortion (in MSE) after
the mode decisions for all of sub8x8 blocks in a 8x8 window are finished,
when the dist-8x8 distortion for luma 8x8 pixels are computed with
new distortion metric.

All the code separating y and uv distortion in inter mode search has
been removed in this commit.

Change-Id: Ieaccb7915df5faeb5e89a7e70b2b7cbac65231af

This patch moves the sb_size to the keyframe header from all frame
headers as discussed in the AOM Codec WG meeting.

Change-Id: Ia0190e57b36b98e51ab61ad4ff14f24f9c294877

Change-Id: I7f2c0bfc12451710622161bd52ff689e642d1e52

Change-Id: Id667950de84ad6a0b55222264a0ce8473cd10bcc

Change-Id: Ib85b2a778e1306444ac0cbbfb212e739a0f0009c

The rd_test_partition3 function basically calls rd_pick_sb_modes on
each of three sub-blocks of a partition (this is one of the mixed size
partitions like PARTITION_HORZ_A) and adds up the cost. There's also
some extra book-keeping to do with stopping early if we know the
partition is more expensive than what we've already seen.

Before this patch, each call and its supporting code was written out
in rd_test_partition3 with lots of repetition (made even more
unreadable because of the #if/#endif blocks). This patch moves the
"per-subblock" logic into a new function, rd_try_subblock, and
replaces the code with calls to that function.

The patch also rewrites the nested conditionals

  if (A) { if (B) { if (C) { X; } } } return;

as

  if (! A) return; if (! B) return; if (! C) return; X;

which means the code for X doesn't need to be so indented.

The patch also uses the new function to replace the bodies of the
PARTITION_HORZ_4 and PARTITION_VERT_4 partition types.

Note that lots of the remaining ugliness is to do with supporting
CONFIG_SUPERTX with ext-partition types. These explicitly aren't
supported together at the moment, so another option would be to rip
out that code entirely.

Change-Id: I9af40d96aa1384f24a088a73a711311638490250

For the first superblock, signal a filter level (6 bit).
For other superblocks, signal the delta (3 bit) and sign (1 bit).
To guarantee the delta can be represented by 3 bits, the search
range of current superblock filter level is capped as
curr_lvl = prev_lvl +- ((1 << 3) - 1)

Change-Id: Ibbe2941aa96ec1220ed2adb7edf3d29cb032f0ec

this was enabled in test.mk in:
c9ac07f6 Re-enable resize test.

Change-Id: I0836e533c77570c5176b2c99fe7a3d19d73ac8af

If dist-8x8 is enabled, it has disabled early-exit during the RD
of sub8x8 tx block in 8x8 or larger partition for plane 0.

However, for the sanity check mode of dist-8x8 which still use MSE,
this can cause different bdrate from the reference (i.e
dist-8x8 off). Hence, if tune-metirc is MSE then
regular early-exit will perform for all cases.

Note that the rd cost used for early-exit condition during tx mode
decision does not include rate for mode itself but only coefficints.
So, when the total distortion for a partition is very small, the rd cost
of mode can exceed that of distortion.

Change-Id: Ie25c64064b78d2d1582dbb0c286212fbc52dbdb9

Function speedup (i7-6700),  sse2 verse C:
Predictor      V_PRED    DC_PRED
4x8            ~1.5x     ~4.9x
8x4            ~2.5x     ~4.8x
8x16           ~1.9x     ~9.1x
16x8           ~1.9x     ~4.4x
16x32          ~2.1x     ~5.8x
32x16          ~2.0x     ~3.6x

Change-Id: I6deffd0637e57ee5d0bd533502f5705148c4cdd4

Change-Id: I24cb257367775608b2b6e4d45d07c30b511af251

Fix the missing the compiler fix in 772cb136.

Change-Id: I2e1d0576964483c7890fb87e3db5451816883b9b

Combine the index and type mapping tables for intra and inter.

Change-Id: I4274f0b5afac26f7e395467bdf5cf18077eacbee

It's not being used anymore, replaced by the CDF model.
The counters are moved under the entropy-stats experiment, so that they
can be used to tune the default CDF values.

Change-Id: I50cb170c4c8699cf7317fd77853b3fedf2342d3e

Use common structure for inter and intra tx type information when
possible.

Change-Id: I1fd3bc86033871ffbcc2b496a31dca00b7d64b31

Fix a compiler error when both DISABLE_TRELLISQ_SEARCH and dist-8x8
are turned on.

Change-Id: Idd122cb4fc391c449bfcd17ec8406d72ef92d15d

(Fixes sanitizer warning)

Change-Id: I7d6d8bbc38adc0a09f646e3e79569c3ca58643e2

Patch a4817a6b added the sub8x8tx_in_gte8x8blk_in_plane0 variable, but
it's read in a block that doesn't exist if DISABLE_TRELLISQ_SEARCH is
true, causing a warning in the overnight "lv-map" unit tests.

Change-Id: I916b61957ffdad1ac7be38efd4051d266d4bd61c

Return 0 when eob is 0. It is possible that when eob is 0, the
fed in quantization matrix is uninitialized in the rate distortion
optimization process.

Change-Id: I2dcf558203bf60ba7332c6bf585a978f7c7309b5

Limit the motion field referencing region to be within a 64x64
coding block region to reduce line buffer requirement. The coding
performance loss numbers are 0.028% for lowres and 0.064% for
midres.

Change-Id: I4fe1a915c5710b763591b7046c45459b02543377

Use integer multipliciation to replace the division operation in
the linear projection in mfmv. The coding performance change is
around 0.01% level.

Change-Id: I901fd3a2b349f1babccb88dbf74f84091079b8d3

This reverts commit 863efacc.

Reason for revert: <INSERT REASONING HERE>

Change-Id: I3d9efef938fe0a2ec4914adc23d0e53b873109d4

This CL fix warning: unused variable
‘sub8x8tx_in_gte8x8blk_in_plane0’
when DISABLE_TRELLISQ_SEARCH is on
Change-Id: I64908513d58287e53ae5fd040e8099089176aace

BUG=aomedia:707

Change-Id: I9b819e630a1c56b5df359c6e65acb6f80ae513e3

Search and pick filter level for each superblock after the entire
frame has been encoded. And then apply loop filtering using
selected filter levels.

Now it can't work with LOOPFILTER_LEVEL. Y, U, V planes share the
same filter level.

Per superblock, filter levels are signaled via bitstream.

Future work: move selecting filter level into each superblock
rather than wait for the whole frame encoded.

Change-Id: I46cb994ec4866abb0456146f7c3cae8813c1d52e

Change-Id: I78bb38e3d61d74442278bd5ed1c4f695b7c8e24a

Make the probability model update system better customized for the
level map coding scheme. This improves the level map coding
performance by 0.2% for lowres and 0.1% for midres.

Change-Id: Ib6d3abb36d50ff7485c4ceb411fe94e8fb060416

While configuring with "--extra-cflags=-DDISABLE_TRELLISQ_SEARCH=1",
there was an unused variable warning. It was fixed.

Change-Id: I1a4df461eb95bc4970f3bbf2e90bac7e394c318e

Add sse4_1 functions for Intra-edge experiment:
  av1_upsample_intra_edge_sse4_1()
  av1_upsample_intra_edge_high_sse4_1()

Approx cycle reduction at qp 20, 1 kf:
  Enc:  0.5% to 0.3%
  Dec:  0.4% to 0.2%

Change-Id: I97f0eee09b78218b418b484d80c338cec037f1b9

the original calculation for super_block_upper_left calculation is
wrong. this flag is true even for 32x32 block boundary. also it is not
consistent with other places where this flag is calculated.

Change-Id: I6b7b613d1f81eb11d27c67491375f6c1a47878e1