Doing tokenize in the last step of RD loop and then doing packing
tokens in bitstream packing phase is hard for debugging.

Therefore, we create a frame-level buffer to store the txfm coeffs
from the reconstruction in RD loop and then in bitstream packing
phase, we can code the txfm coeffs directly.

Change-Id: I999470eef6e038317a91585df2bdfc20aca3573e

Skip redundant computation of recon_variance value.

Change-Id: I0ea1a3528f8bdc7095d9cc458ce0bd3a958989b4

Skip the 2x2 uv block rate-distortion optimization process in the
non-reference positions in key frame encoding.

Change-Id: I51112492fd79f9a1c205fba151f8658cea9ee85f

vs2013 is still supported, it still uses the old behavior which supports
inline in c++ only.

Change-Id: I74417c6c93774e55122bbbf1db0283e2c30fe63e

This commit adds a new experiment, Daala's distortion function,
which is designed to better approximate perceptual distortion
in 8x8 pixel blocks.

This experiment is expected to work best with PVQ.

It measures the variance of overlapped 4x4 regions in the 8x8 area,
then uses these variances to scale the MSE of weighted frequency domain
distortion of 8x8 block.

Since AV1 calculates distortion in blocks as small as 4x4, it is not possible to
directly replace the existing distortion functions of AV1,
such as dist_block() and block_rd_txf().
Hence, there has been substantial changes in order to apply
Daala's 8x8 distortion function.
The daala distortion function is applied
after all 4x4 tx blocks in a 8x8 block are encoded (during RDO),
as in below two cases:
1) intra/inter sub8x8 predictions and
2) 4x4 transform with prediction size >= 8.

To enable this experiment, add '--enable-daala-dist' with configure.

TODO: Significant tuning of parameters is required since the function has
originally came from Daala thus most parameters would not work
correctly outside Daala.
The fact that chroma distortion is added to the distortion of AV1's RDO is
also critical since Daala's distortion function is applied to luma only
and chroma continues to use MSE.

Change-Id: If35fdd3aec7efe401f351ba1c99891ad57a3d957

These are under EXT_TX + RECT_TX experiment combo.

Results
=======

Derf Set:
--------
All Intra frames: 1.8% avg improvement (and 1.78% BD-rate improvement)
Video: 0.230% avg improvement (and 0.262% BD-rate improvement)

Objective-1-fast set
--------------------
Video: 0.52 PSNR improvement

Change-Id: I1893465929858e38419f327752dc61c19b96b997

Replace hard coded 4x4 transform block step size assumption with
scalable table access.

Change-Id: Ib1cc555c2641e5634acdd91ca33217f00aeb0b89

Change-Id: I6ce654b582f2a9d45a40bf22ba597b47d418a0be

Change-Id: I0e1954ca281df60e56c3b1a5dcd54c47c4256649

PVQ replaces the scalar quantizer and coefficient coding with a new
design originally developed in Daala. It currently depends on the
Daala entropy coder although it could be adapted to work with another
entropy coder if needed:
./configure --enable-experimental --enable-daala_ec --enable-pvq

The version of PVQ in this commit is adapted from the following
revision of Daala:
https://github.com/xiph/daala/commit/fb51c1ade6a31b668a0157d89de8f0a4493162a8

More information about PVQ:
- https://people.xiph.org/~jm/daala/pvq_demo/
- https://jmvalin.ca/papers/spie_pvq.pdf

The following files are copied as-is from Daala with minimal
adaptations, therefore we disable clang-format on those files
to make it easier to synchronize the AV1 and Daala codebases in the future:
 av1/common/generic_code.c
 av1/common/generic_code.h
 av1/common/laplace_tables.c
 av1/common/partition.c
 av1/common/partition.h
 av1/common/pvq.c
 av1/common/pvq.h
 av1/common/state.c
 av1/common/state.h
 av1/common/zigzag.h
 av1/common/zigzag16.c
 av1/common/zigzag32.c
 av1/common/zigzag4.c
 av1/common/zigzag64.c
 av1/common/zigzag8.c
 av1/decoder/decint.h
 av1/decoder/generic_decoder.c
 av1/decoder/laplace_decoder.c
 av1/decoder/pvq_decoder.c
 av1/decoder/pvq_decoder.h
 av1/encoder/daala_compat_enc.c
 av1/encoder/encint.h
 av1/encoder/generic_encoder.c
 av1/encoder/laplace_encoder.c
 av1/encoder/pvq_encoder.c
 av1/encoder/pvq_encoder.h

Known issues:
- Lossless mode is not supported, '--lossless=1' will give the same result as
'--end-usage=q --cq-level=1'.
- High bit depth is not supported by PVQ.

Change-Id: I1ae0d6517b87f4c1ccea944b2e12dc906979f25e

PVQ replaces the scalar quantizer and coefficient coding with a new
design originally developed in Daala. It currently depends on the
Daala entropy coder although it could be adapted to work with another
entropy coder if needed:
./configure --enable-experimental --enable-daala_ec --enable-pvq

The version of PVQ in this commit is adapted from the following
revision of Daala:
https://github.com/xiph/daala/commit/fb51c1ade6a31b668a0157d89de8f0a4493162a8

More information about PVQ:
- https://people.xiph.org/~jm/daala/pvq_demo/
- https://jmvalin.ca/papers/spie_pvq.pdf

The following files are copied as-is from Daala with minimal
adaptations, therefore we disable clang-format on those files
to make it easier to synchronize the AV1 and Daala codebases in the future:
 av1/common/generic_code.c
 av1/common/generic_code.h
 av1/common/laplace_tables.c
 av1/common/partition.c
 av1/common/partition.h
 av1/common/pvq.c
 av1/common/pvq.h
 av1/common/state.c
 av1/common/state.h
 av1/common/zigzag.h
 av1/common/zigzag16.c
 av1/common/zigzag32.c
 av1/common/zigzag4.c
 av1/common/zigzag64.c
 av1/common/zigzag8.c
 av1/decoder/decint.h
 av1/decoder/generic_decoder.c
 av1/decoder/laplace_decoder.c
 av1/decoder/pvq_decoder.c
 av1/decoder/pvq_decoder.h
 av1/encoder/daala_compat_enc.c
 av1/encoder/encint.h
 av1/encoder/generic_encoder.c
 av1/encoder/laplace_encoder.c
 av1/encoder/pvq_encoder.c
 av1/encoder/pvq_encoder.h

Known issues:
- Lossless mode is not supported, '--lossless=1' will give the same result as
'--end-usage=q --cq-level=1'.
- High bit depth is not supported by PVQ.

Change-Id: I1ae0d6517b87f4c1ccea944b2e12dc906979f25e

It is no longer used anywhere.

Change-Id: I5d40664373d66821c5382f6155302b8719ce58c0

Seems to be dead code

Change-Id: I17b3edc9e82d6a1da172a686522358a6b1a630e9

Previously, we assumed that av1_init_plane_quantizers is always called with
segment_id == xd->mi[0]->mbmi.segment_id (and use the latter to derive the value
of 'qindex' to use in the quantizer). But this is no longer true when supertx
is enabled. This patch instead remembers the value of 'qindex' derived from
the latest call to av1_init_plane_quantizers and uses that directly.

Change-Id: Ifa1c5bf74cad29942ff79b88ca92c231bc07f336

Check the encoding statistics. If all the coding blocks use the
max transform size, skip transform size coding in the frame header.

Change-Id: I31cb16314e87f945d7e95a34a90a5536b3ed82d5

While we are at it:
- Rename some variables to more meaningful names
- Reuse some common consts from a header instead of redefining them.

Cherry-picked from aomedia/master: 863b0499

Change-Id: Ida5de713156dc0126a27f90fdd36d29a398a3c88

This is not used since the commint 00cd5de5,
"Remove skip_recode speed feature".

Change-Id: Ic03da6c0095f6285a3889d5d22e8aaa2e6cbfd79

To get ready for pulling AV1 to nextgenv2. Refactoring is done to
make the code structures similar, especially for the motion search
part.

Change-Id: I5d7636394408d97de55394d668540f5627827983

This flag was already added to aomedia/master, so bringing it back to
webm/nextgenv2, as part of an effort to get the two codebases in sync.

Change-Id: I2b933a6a160e4210d1411a9e7978149eb8553205

This commit renames LIBVPX_TEST_DATA_PATH to LIBAOM_TEST_DATA_PATH,
with a work around for working with jenkins environmnet variables.

Change-Id: If664ce57e25ad2af8121d1b578bf64043f0baa2a

Conflicts:
	av1/encoder/block.h
	av1/encoder/encodeframe.c

Change-Id: I941e7b9e76380f262b173928d3c5132c5613b3ce

Change-Id: I134d276234b3b8aa7df1ab647892b5d739647f4c

This is a port of
https://chromium.googlesource.com/webm/libvpx/+/92922be83c0b4b98d50f24cc3a1e7cc0a1b5741a
from nextgenv2

Change-Id: Ie62170f7305a1588a2d170e36297618a549f1787

Change-Id: I2b2b70e756b7eb9611b7b33b7d5f19b3b30e0a50

Cherry-Picked the following commits:
0defd8f2 Changed "WebM" to "AOMedia" & "webm" to "aomedia"
54e66767 Replace "VPx" by "AVx"
5082a369 Change "Vpx" to "Avx"
7df44f17 Replace "Vp9" w/ "Av1"
967f722f Remove kVp9CodecId
828f30ce Change "Vp8" to "AOM"
030b5ffc AUTHORS regenerated
2524caee Add ref-mv experimental flag
016762be Change copyright notice to AOMedia form
81e55269 Replace vp9 w/ av1
9b94565b Add missing files
fa8ca9f2 Change "vp9" to "av1"
ec838b76  Convert "vp8" to "aom"
80edfa01 Change "VP9" to "AV1"
d1a11fb9 Change "vp8" to "aom"
7b582513 Point to WebM test data
dd1a5c8d Replace "VP8" with "AOM"
ff00fc0f Change "VPX" to "AOM"
01dee0bb Change "vp10" to "av1" in source code
cebe6f0c Convert "vpx" to "aom"
17b05679 rename vp10*.mk to av1_*.mk
fe5f8a8a rename files vp10_* to av1_*

Change-Id: I6fc3d18eb11fc171e46140c836ad5339cf6c9419

Manually cherry-picked commits:
ceef0583 libvpx->libaom part2
3d26d916 libvpx -> libaom
cfea7dd7 vp10/ -> av1/
3a8eff73 Fix a build issue for a test
bf4202ed Rename vpx to aom

Change-Id: I1b0eb5a40796e3aaf41c58984b4229a439a597dc

This was causing a compiler warning from -Wcast-qual.

Change-Id: Ie525ffe20be4f38ced68fb0c4141e36400eb0717

clang-format authored Aug 11, 2016 and James Zern committed Aug 12, 2016

Change-Id: I58a42ced5b8a4338524434ff3356850b89aa705a

- Avoid some memcpy()s
- Remove indices array
- Make pre_indices array local
- Avoid rounding twice
- Other small simplifications

Change-Id: Iac3236daaad04f21f54054cdd9504de13b942a07

We have renamed following Macros to avoid name confusion:

REFS_PER_FRAME --> INTER_REFS_PER_FRAME
(= ALTREF_FRAME - LAST_FRAME + 1)
MAX_REF_FRAMES --> TOTAL_REFS_PER_FRAME
(= ALTREF_FRAME - INTRA_FRAME + 1)

INTER_REFS_PER_FRAME specifies the maximum number of reference frames
that each Inter frame may use.
TOTAL_REFS_PER_FRAME is equal to INTER_REFS_PER_FRAME + 1, which
counts the INTRA_FRAME.

Further, at the encoder side, since REF_FRAMES specifies the maximum
number of the reference frames that the encoder may store, REF_FRAMES
is usually larger than INTER_REFS_PER_FRAME. For example, in the
ext-refs experiment, REF_FRAMES == 8, which allows the encoder to
store maximum 8 reference frames in the buffer, but
INTER_REFS_PER_FRAME equals to 6, which allows each Inter frame may
use up to 6 frames out of the 8 buffered frames as its references.
Hence, in order to explore the possibility to store more reference
frames in future patches, we modified a couple of array sizes to
accomodate the case that the number of buffered reference frames is
not always equal to the number of the references that are being used
by each Inter frame.

Change-Id: I19e42ef608946cc76ebfd3e965a05f4b9b93a0b3

Keep track of the best and second best full pixel motion vector
candidates, and do subpel search around both of them.

Compression improvement:
lowres 0.22%   midres 0.23%   hdres 0.18%

No noticeable encoding speed changes observed on lowres test clips.

Change-Id: I5f4df2a03d1db061cfdfdba6138b27e9ea91f089

from PICK_MODE_CONTEXT and MACROBLOCK

Change-Id: I42f98ce51871948244bdcaaaeb3d0191622116ae

1. Add "best_mv" in MACROBLOCK to store the best motion vector
during motion search, so that we don't need to pass its pointer
to various motion search functions.

2. Declare some functions as static when possible.

3. Fix some indents.

Change-Id: I0778146c0866cbc55e245988c59222577ea8260e

Change-Id: I22f1299545d4c75d80e72d479be66f66ea142ef1

Commit 0d6980d7 removed some use
of the skip_txfm optimization, and the rest are not productive.

The current use of this optimization is only used with --good
and --cpu-used >= 3, however the overhead of this is higher than the
speedup it yields.

Removing this, and subsequently simplifying model_rd_for_sb yields
a net encoder speedup:
--cpu-used=0    ~1.5% faster
--cpu-used=3    ~2.0% faster

The code simplification is also significant.

Change-Id: I1dd668c32de15a2e912c59c42379d0f9e1032ff8

Add the ability to pick between 3 quantization profiles.
The profile is chosen based on the entropy context at the
block level.

Change-Id: Iaea0485798441b7d635962c2563f3a477f582dac

pred_variance in obmc experiment is equivalant to recon_variance in
baseline

Change-Id: Iba8fb9bd973898be5a0d87a507ceaf65c75bdc51

This is purely refactoring in preparation of fixing supertx segment
handling

Change-Id: I74bcae34241fdf2b592e1cd45b67af77b9e16c9a

This experiment implements non-uniform quantization where
the width of the bins increases gradually to more closely
match a laplacian distribution of the coeficcients.

Performance Gain:
derflr: 0.15%
hevcmr: 0.675%

Change-Id: I25234244e3bcd94b87c1f77cf682190b61c8ef94

The assumption doesn't hold true in the current codebase. Remove
this speed feature to simplify the codebase.

Change-Id: I9b69f484c9b7cd612b825047cc5b2fce63ee0af7