This commit makes the encoder to properly account for all transform
block sizes when combining statistics from encoding threads.

Change-Id: I010acd3b247dc890f63756d3d1436b1fb52ea2d9

Change-Id: I4c4300f3f565d8aecf65669b77aaa874bb73a3a0

Also includes some minor renaming of macros.

Change-Id: I9493cc97c6ec9c8dae8020a05a02d6f322db9a02

Change-Id: Ic2fbc66e9212da32930c6a8ba1a749e3a37c5b9a

Includes:
Some cleanups/refactoring
Better buffer management.
Some preps for future chrominance restoration.

Change-Id: Ia264b8989b5f4a53c0764ed3e8258ddc212723fc

This is intended to simplify hardware and multithreaded
implementations.

Change-Id: I6aa95b67c03b794a0f3d5cf2f65c576d05f2ca7d

A crash was reported while encoding >1080p file using best mode.
(aom issue 107 and issue 108). The cause was that the speed features
were initialized using good mode in AV1. The patch(commit f330ca)
disabled the upsampled reference usage only for good mode, but not for
best mode. So, for 1st key frame, the encoder incorrectly called
reset_use_upsampled_references() that caused the segfault.

The patch(commit e4e93b) disabled the upsampled reference usage for all
encoding modes and resolved the issue. This patch fixed issue 108 and
further made sure the upsampled references would be used correctly.

BUG=aomedia:108

Change-Id: I8362cc7cd82be324fd77f0bec315aeba509903bd

Uses number of 4x4 blocks using global motion as metric to
decide whether to recode or not.

Improves performance a little.
lowres: -0.658%

Change-Id: I6b1913b7bd31d7a25ca8ca5c75c9b6bb87151a78

With this patch affine or homography models can be enabled
by simply changing the value of the GLOBAL_TRANS_TYPES
macro in common/mv.h to 4 and 5 respectively. Currently
it is left at supporting only rotzoom. There is a small
gain with enabling affine.

Also refactors costing to change based on the model type.

Change-Id: I46c1759de06c42c176c64ec21307ff347ddcc259

The commit fixes the segfault shown in  test AV1/ActiveMapTestLarge
when build with the following config:
../../aom/configure --disable-install-docs --enable-unit-tests
--enable-debug --disable-optimizations --enable-aom-highbitdepth
--enable-experimental --enable-ans --enable-bidir-pred
--enable-dual-filter --enable-entropy --enable-ext-inter
--enable-ext-interp --enable-ext-intra --enable-ext-partition
--enable-ext-partition-types --enable-ext-refs --enable-ext-tx
--enable-loop-restoration --enable-filter-intra --enable-motion-var
--enable-new-quant --enable-ref-mv --enable-supertx --enable-var-tx

BUG=aomedia:99

Change-Id: I900d5e1d5458c482a1f0a3dde769c1ad93acdca7

Change-Id: I73bcb8ab5727e2d07e34ca35e9e014f3c6f63d56

BUG=aomedia:99

Change-Id: I36c3c3728500e0eefb7c56720a467e8754b85009

Change-Id: I9b2465807c2dd6e356807f3f3555394b382918ad

Adds 64x64 default scan orders
Adds 64x64 coefficient entropy models (just copy the 32x32 ones)
Entropy context updates for 64x64 transform
Various misc. changes to support 64x64 transforms

Change-Id: I2c0bc4ba540886dd196e87a78b205407ad3e866b

Change-Id: Iad227b07b6c3f5beebef971cabec787b01e6f51f

Adds a feature to recode if global motion is used for a reference but
has very few blocks in the frame actually using it.

lowres improves to -0.512% on average.

Change-Id: I61a36770e1b7103b9a27706909443c3f14ee4e42

Change-Id: Ie57676814833824b1f3612b60ed9b0485c7ffd52

Default MTU size is 0, which implies a fixed number
of tile groups. MTU matching overrides fixed tile
group. MTU matching will succeed unless a single
tile is bigger than the MTU.

--mtu-size is in bytes, and includes headers but
not transport/wrapper overheads

Change-Id: I2b70bd41b175b54273b02d836f2a84011f617a7c

Change-Id: Iaaf9244678cde74cf3e216f3b8c9d293f206f27e

PVQ replaces the scalar quantizer and coefficient coding with a new
design originally developed in Daala. It currently depends on the
Daala entropy coder although it could be adapted to work with another
entropy coder if needed:
./configure --enable-experimental --enable-daala_ec --enable-pvq

The version of PVQ in this commit is adapted from the following
revision of Daala:
https://github.com/xiph/daala/commit/fb51c1ade6a31b668a0157d89de8f0a4493162a8

More information about PVQ:
- https://people.xiph.org/~jm/daala/pvq_demo/
- https://jmvalin.ca/papers/spie_pvq.pdf

The following files are copied as-is from Daala with minimal
adaptations, therefore we disable clang-format on those files
to make it easier to synchronize the AV1 and Daala codebases in the future:
 av1/common/generic_code.c
 av1/common/generic_code.h
 av1/common/laplace_tables.c
 av1/common/partition.c
 av1/common/partition.h
 av1/common/pvq.c
 av1/common/pvq.h
 av1/common/state.c
 av1/common/state.h
 av1/common/zigzag.h
 av1/common/zigzag16.c
 av1/common/zigzag32.c
 av1/common/zigzag4.c
 av1/common/zigzag64.c
 av1/common/zigzag8.c
 av1/decoder/decint.h
 av1/decoder/generic_decoder.c
 av1/decoder/laplace_decoder.c
 av1/decoder/pvq_decoder.c
 av1/decoder/pvq_decoder.h
 av1/encoder/daala_compat_enc.c
 av1/encoder/encint.h
 av1/encoder/generic_encoder.c
 av1/encoder/laplace_encoder.c
 av1/encoder/pvq_encoder.c
 av1/encoder/pvq_encoder.h

Known issues:
- Lossless mode is not supported, '--lossless=1' will give the same result as
'--end-usage=q --cq-level=1'.
- High bit depth is not supported by PVQ.

Change-Id: I1ae0d6517b87f4c1ccea944b2e12dc906979f25e

PVQ replaces the scalar quantizer and coefficient coding with a new
design originally developed in Daala. It currently depends on the
Daala entropy coder although it could be adapted to work with another
entropy coder if needed:
./configure --enable-experimental --enable-daala_ec --enable-pvq

The version of PVQ in this commit is adapted from the following
revision of Daala:
https://github.com/xiph/daala/commit/fb51c1ade6a31b668a0157d89de8f0a4493162a8

More information about PVQ:
- https://people.xiph.org/~jm/daala/pvq_demo/
- https://jmvalin.ca/papers/spie_pvq.pdf

The following files are copied as-is from Daala with minimal
adaptations, therefore we disable clang-format on those files
to make it easier to synchronize the AV1 and Daala codebases in the future:
 av1/common/generic_code.c
 av1/common/generic_code.h
 av1/common/laplace_tables.c
 av1/common/partition.c
 av1/common/partition.h
 av1/common/pvq.c
 av1/common/pvq.h
 av1/common/state.c
 av1/common/state.h
 av1/common/zigzag.h
 av1/common/zigzag16.c
 av1/common/zigzag32.c
 av1/common/zigzag4.c
 av1/common/zigzag64.c
 av1/common/zigzag8.c
 av1/decoder/decint.h
 av1/decoder/generic_decoder.c
 av1/decoder/laplace_decoder.c
 av1/decoder/pvq_decoder.c
 av1/decoder/pvq_decoder.h
 av1/encoder/daala_compat_enc.c
 av1/encoder/encint.h
 av1/encoder/generic_encoder.c
 av1/encoder/laplace_encoder.c
 av1/encoder/pvq_encoder.c
 av1/encoder/pvq_encoder.h

Known issues:
- Lossless mode is not supported, '--lossless=1' will give the same result as
'--end-usage=q --cq-level=1'.
- High bit depth is not supported by PVQ.

Change-Id: I1ae0d6517b87f4c1ccea944b2e12dc906979f25e

Low latency:
PSNR YCbCr:     -0.15%      0.11%      0.12%
   PSNRHVS:     -0.25%
      SSIM:     -0.26%
    MSSSIM:     -0.26%
 CIEDE2000:     -0.03%

High latency:
PSNR YCbCr:     -0.18%      0.18%      0.07%
   PSNRHVS:     -0.20%
      SSIM:     -0.21%
    MSSSIM:     -0.21%
 CIEDE2000:     -0.03%

Change-Id: Ieb86d9ba353220de6454bdc15cea825944b6385b

This prevents a crash if the upsample_refs speed feature is
changed as part of set_size_dependent_vars, when the recode
loop is enabled.

Change-Id: I645e389bfe961879dd2001439a34fde2993868d9

This prevents a crash if the upsample_refs speed feature is
changed as part of set_size_dependent_vars, when the recode
loop is enabled.

Change-Id: I645e389bfe961879dd2001439a34fde2993868d9

This is a manual adaptation of the following commit from aom/master:
ce12003d

The original commit message:

A tile group is a set of tiles in scan order.

Each tile group has a version of uncompressed and compressed headers,
identical apart from tile group parameters.
Encoding probability updates takes account of the number of
headers to control overheads.

The decoder supports arbitrary numbers of tile groups with
arbitrary number of tiles. The number of tiles in a TG is
signalled in the uncompressed header for that TG.

The encoder currently only supports a fixed number
of TGs (3, when error resilient mode is on) of equal size
(except possibly for the last one).

The average BDR performnce with 3 tile groups versus
anchor with error resilient mode and up to 16 tiles is:

NR YCbCr:      3.02%      3.04%      3.05%
PSNRHVS:      3.09%
SSIM:      3.06%
MSSSIM:      3.05%
CIEDE2000:      3.04%

Change-Id: I9b97c5ed733103b9160a3a5d4370de5322c00c0b

Imported changes from VP9:
https://chromium-review.googlesource.com/#/c/402551/
https://chromium-review.googlesource.com/#/c/403128/

Change-Id: I8570c867190a6fa641926431ce97f7d9d7da3528

Seems to be dead code

Change-Id: I17b3edc9e82d6a1da172a686522358a6b1a630e9

This will allow for aligned cdfs and scan orders inside.

Change-Id: I8ebcd64d55e41da20f518a39ae6ef192def70109

This reduces some memcpys and callocs.

Cherry-picked from aomedia/master: 40810138

Change-Id: If04580af4c63892c8af8ac5b405c7d6aabe5af89

Plus a small code clean up. The experiment of EXT_REFS, compared against
the baseline, using Overall PSNR, now obtains a gain on lowres as:
Avg: -5.818; BDRate: -5.653

Compared against the previous EXT_REFS results on lowres, a tiny gain is
obtained as:
Avg: -0.047, BDRate: -0.063

(1) 780952 Add encoder first pass support to bi-prediction in EXT_REFS
(2) f91498 Add pred prob handling for new references in EXT_REFS
(3) e91472 Add decoder support for bi-direct prediction in EXT_REFS
(4) 0dbac9 Add encoder support to new references in EXT_REFS
(5) ad70cc Remove hard-coded number for EXT_REFS
(6) 9c1e2f Add the use of new reference frames at encoder in EXT_REFS
(7) 6d4fde Add the experiment flag of EXT_REFS

Change-Id: I26f7ca45b9ede7579fdb9d0d6a1a91f4334599bd

While we are at it:
- Rename some variables to more meaningful names
- Reuse some common consts from a header instead of redefining them.

Cherry-picked from aomedia/master: 863b0499

Change-Id: Ida5de713156dc0126a27f90fdd36d29a398a3c88

In the header, all we need is number of stat types, not the names for actual
types.

Removing it avoids names like 'Y', 'U', 'V' and 'ALL' being visible
in all files that include the encoder.h header.

Change-Id: I874a73a3cfe6bcb29aedea102077a52addc49af6

This array was allocated and used to save and restore segmentation map,
however the original segmentation map was never modified between the
calls to save and restore.

Change-Id: Iaf0fbfed733c097e84cf44d2aa6b8f35d2fb456b

This flag was already added to aomedia/master, so bringing it back to
webm/nextgenv2, as part of an effort to get the two codebases in sync.

Change-Id: I2b933a6a160e4210d1411a9e7978149eb8553205

This reverts commit 9b25f306 to
reinstate the reverted commit with fixes that solved the build issues
when --enalbe-clpf is used in configure.

Change-Id: I15447cae7fa9b3deb27976345dc3db230a4a7a60

This reverts commit 97535038.

Change-Id: I9f8e891739352ca2bde4b294e37c85a668f416e0

These signals were in the uncompressed frame header (as a temporary
hack), which caused two problems:

* We don't want that header to be duplicated in the slice header
* It was necessary to signal the number of bits to transmit up front

However, the filter size can be 128x128 which is greater than the SB
size, and a decoder wouldn't be able to know whether to read a bit or
not until the final SB of that 128x128 block has been decoded
(depending on whether the 128x128 is all skip or not).  Therefore the
signalling was changed for 128x128 blocks so that every top left SB of
a 128x128 filter block contains a signal regardless of whether the
block is all skip or not.  Also, all the MB's of 128x128 block are
filtered even if they are skip MB's.  This gives the signal a purpose
even when the 128x128 block is all skip, and it also gives a slight
coding gain as it leaves a way to filter skip blocks, which was
previously forbidden.

Low latency:
PSNR YCbCr:     -0.19%     -0.14%     -0.06%
   PSNRHVS:     -0.15%
      SSIM:     -0.13%
    MSSSIM:     -0.15%
 CIEDE2000:     -0.19%

High latency:
PSNR YCbCr:     -0.03%     -0.01%     -0.09%
   PSNRHVS:      0.04%
      SSIM:      0.00%
    MSSSIM:      0.02%
 CIEDE2000:     -0.02%

Change-Id: I69ba7144d07d388b4f0968f6a53558f480979171

To get ready for pulling AV1 to nextgenv2
Replace the experimental flag by MOTION_VAR. Rename major variables.

Change-Id: If6cf4f37b9319c46d8f90df551cc7295d66ca205

Currently the patch does not have any impact on the RD performance. The
fix could however potentially help on the next step of work, especially
when the extra altref frames allow non-zero temporal filtering strength
and their corresponding OVERLAY frames, i.e. the INTNL_OVERLAY frames
are being added.

Change-Id: I2e07fb3d0aa547a0b5dd05bb4ba865cd46309076

PSNR YCbCr:     -0.01%     -0.06%     -0.17%
   PSNRHVS:      0.01%
      SSIM:      0.03%
    MSSSIM:      0.00%
 CIEDE2000:     -0.05%

Change-Id: I1205c021bfc5cee6f80344fec92aabb529af9bd1