Use cumulative range check to avoid the division in the for-loop
for base range decoding in lv-map. This makes the lv-map transform
coefficient decoding process 10% faster at the decoder end, at
high target bit-rate range.

Change-Id: I85e9646ce05f0c20c10f776ff9cdd082b16e9989

When the coding block size is below 16x16, the current tx size
search loop may end up searching 4x4 transform size multiple times.
This commit skips the redundant search steps. Tested a a few
1080p clips at 4 Mbps, the encoding speed is up by 10% for key
frame coding, and 3% for overall.

Change-Id: Ifbb27bad1ec55ef0011b1e789d995f36b3f4605e

Includes unit tests for conformance and speed.

SSSE3/CFLPredictTest:
4x4: C time = 2063 us, SIMD time = 313 us (~6.6x)
8x8: C time = 6656 us, SIMD time = 493 us (~14x)
16x16: C time = 24970 us, SIMD time = 1327 us (~19x)
32x32: C time = 59020 us, SIMD time = 5178 us (~11x)

AVX2/CFLPredictTest:
4x4: C time = 2052 us, SIMD time = 333 us (~6.2x)
8x8: C time = 6712 us, SIMD time = 513 us (~13x)
16x16: C time = 25292 us, SIMD time = 1023 us (~25x)
32x32: C time = 58994 us, SIMD time = 2828 us (~21x)

Change-Id: I08690a548be981ff10e184de468b9e0e691ee812

Change-Id: I03c6cf0b58f9e1d74c7d30d76e846be9a640dab7

Designs the sequence header syntax to support various
profiles as decided in the AV1 codec WG:

Profile 0: 4:2:0 and 4:0:0 8/10-bit
Profile 1: 4:4:4 8/10-bit
Profile 2: 4:2:0/4:0:0/4:4:4 12-bit as well as 4:2:2 8/10/12-bit

Change-Id: Iea351698280e37d65847bf75a43c5bbeba8f7cf4

This issue isn't triggered currently though, because encode_block() is
always called with enable_optimize_b = 1.

Change-Id: I1ce5a35222cb6a647418bf7e4dc3189db5aa3854

Also, make them 'const' when possible.

Change-Id: I871709b6a839405127be7ca6a3d89f4543794ce4

Change-Id: I20df9de1f72f3ccf31ce461fb3eeea3c174c9d83

This experiment has been adopted, we can simplify the code
by dropping the associated preprocessor conditionals.

Change-Id: I3e08eec385c40360e3934fa4f66f7c671e860517

Set up the identity scale factor once per frame, instead of per joint
motion search.

Change-Id: I5edd264655254f4b6a0a923d8ab43adbca044228

These functions are only used in a intra mode context.
Add related sanity check.

Change-Id: I068d9251bd17b621a0d899bbbea01df53f0cfb6d

Add tests of expected failure conditions, and handle invalid pad
sizes in the fixed length encode function.

BUG=aomedia:1125

Change-Id: I8b9d0fbc44bb776bca3b4c3f0467a2125f13be2e

Commit https://aomedia-review.googlesource.com/c/aom/+/39985
changed the deblock filter for subsampled planes, so that
we always look at the bottom-right mi unit in each 8x8 luma
pixel unit. This fixed some issues around determining which
blocks are inter/skip/etc.

However, this introduced a different bug - the boundary info
("are we allowed to filter across the top/left edge of this block?")
is stored in the *top-left* mi unit in each 8x8 luma pixel unit.
But now we we're looking at the bottom-right unit, which always says
it's fine to filter across the top and left boundaries! Thus the
deblock filter effectively ignored the
loop_filter_across_tiles_enabled flag(s).

Fix this issue by looking up the boundary info from the top-left
block in each unit; the other information is still taken from
the bottom-right.

Change-Id: I626ea3358563641a2532ee9c5968fb54bdc78e34

Since 1193 is now fixed.

BUG=aomedia:1193

Change-Id: I79952e42679b68d8a1abc873b2c6f66711a7c4d8

Change-Id: I22d0fac0d8e94fc02a7adf13b144ed17097ff84b

For 2:1 txfm, move the sqrt2 scaling to the end of the fwd txfm
function

This only affects the txfm accuracy slightly but simplifies the
range management a lot.

Change-Id: I2eef8c15b7e8958a1fb0f2ae632604fdbab86046

Change-Id: Ic5b09169d7fbb28d09008756ec71703ff7322d78

The coding block processing is streamlined in cb4x4. There is no
longer a need to do special handle for sub8x8 blocks. This commit
removes the unnecessary memory allocation to the leaf nodes,
corresponding to 4x4 block size. It saves around 20% of the encoder
memory requirement.

Change-Id: I61301d6dea6c0a10ee232d166eea28578e2b3a58

Change-Id: I87b11ec892fe5048f858869a3b6e68d0fe78ea43

dq_type now takes 4 possible values, one of which is
the original multiply-only dequantizer.

Change-Id: I2180f632f0d7f91dc391a6e802a93a77de789228

When a transform block is skipped, reset the txb_entropy_ctx value
to be 0. This clears the assertion failure cases in txk-sel.

Change-Id: I23816761d4383aa9877bc54b117f54d69e4949a9

Update intrabc_cdf per superblock during encoding.
Also added support for entropy_stats.

Change-Id: Ie106309a8932b86bb069990db0b0ea13a4c5e8ae

Change-Id: I0f0cdf479091c308594a0288a0546dbdb76e4450

When --enable-coefficient-range-checking isn't specified,
clamp the coefficient at each stage.

The decoder behaviour is unchanged for valid existing AV1 streams.
However, some AV1 bitstreams that would have been rejected by the
decoder as illegal (range check failure) are now legal bitstreams.

There is no impact on video quality for valid bitstreams,
only bitstreams that would have been considered invalid are affected,
and are now considered as valid ones.

BUG=aomedia:30

Change-Id: Ie72ceeb3cb0561e8d3259767b209a4ef3be90438

The OBU experiment has removed the parsing of super frame indices, so
the test is obsolete when OBU is enabled.

BUG=aomedia:1213

Change-Id: If7d1873f3f41fa9146f9451687eda92b39b035a5

This is the first phase of a new-quant cleanup. We are now only
using a variable width zero bin, so we can simplify the
implementation.

Change-Id: I4da6c663a9d40b4fd08c14818cf7d7fe9b75844c

Add speed feature that uses hash tables to
reuse previously found optimized coefficients
in av1_optimize_txb. This skips some expensive
optimize_txb calls.

Currently shows no significant quality
degredation or speed improvement, and set to off
by default. Requires hash_me, lv_map and
lv_map_multi. Adding to speed features required
changing AV1_COMMON *cm to AV1_COMP *cpi in a
chain of functions.

Variations that have been tried:
-varying the maximum eob on which the feature
activates: 16, 32, 64. 16 currently used. 64
has best hit rate but longer execution time.
-varying the data hashed and the length of hashes
(first hash is 16 bit and based on context data,
while second hash is 16 bit and based only on
pre-optimized qcoeff values.)
-softening the data used for the hashes: ideally
this would raise the number of hits, without
compromising quality too much.

Change-Id: I94f22be82f3a46637c0489d512f2e334a307575f

When current coeff is found to be set as eob, nz_rd is not needed
in the computation of rdcost. Skip it.

Similarly, when has_nz_tail >= 2, computation for nz_rd should be
skipped.

Local runs show ~3% speed up.

Change-Id: I176abf1d1043cab40788934b70558b816af72732

Change-Id: Ied3252c3075e84252d8268a13d3f50066b3418e1

BUG=aomedia:1203

Change-Id: I03ca925734535c31a1ed6bdeb7f29b5b5d5f476c

BUG=aomedia:1195

Change-Id: Ib643ed9d3a23b83e91d908940f5777f6d13b8b19

BUG=aomedia:1209

Change-Id: If4ba99d6784c55768af57cac2b197db62ea2317e

Change-Id: I31a6a66753e1e06a26ad798415c5994920bf22b7

BUG=aomedia:1154

Change-Id: I0c856564dc1eadd9e22be4de6fca5c2b0344dcb5

Use assert to test the assumption noted in the comment.

Ran some encodes with varying per-frame resolutions (using
SUPERRES_RANDOM and RESIZE_RANDOM modes) to ensure that these
asserts are indeed valid.

Change-Id: I30021fff05f9793f80e42c5319bed02b0ae04cbd

BUG=aomedia:1125

Change-Id: I383c752cdd010001e8226ca510097ef693e486bb

Add the last scan position in the 8x32 iscan table. This point
would cause the encoder to ignore the last non-zero coefficient
at scan index 255, hence causing enc/dec mismatch. This commit
fixes and solves this issue.

BUG=aomedia:1193

Change-Id: Ia41ba31ef8549f8809cea3d76ecd91da77e130cf

Put loops of searching jnt_comp only inside handle_inter_mode
function.

Change-Id: I9508dea64da7e6dfe31fad09aac8679cc855f55f

Avoid potential out of boundary buffer access issue.

Change-Id: Ic3766cd5bc2c1c8b4e1cbf38310fe6689aa38b59

When the reference frame size differs from the current coding
frame, skip the use of its motion information for motion field
estimation.

BUG=aomedia:1194

Change-Id: I1a32395367228f4ac86a21099b6b7b2c25e60270