This reverts Change-Id: Ie11dd055255d200954b704b8c2ad8ca3dff7bf5c

BUG=aomedia:1305

Change-Id: I6894928dcadc99a79417034a7096a215693a46f2

Reduce the length of inter_tx_size[] from 1024 to 16.

On a cif test sequence,
encoder memory consumption decreases by 18% (380MB -> 312MB);
decoder memory consumption decreases by 56% (21.4MB -> 9.4MB).

Change-Id: Ie11dd055255d200954b704b8c2ad8ca3dff7bf5c

Enabling CONFIG_MISMATCH_DEBUG with highbd streams was producing undefined
behaviour. This patch adds support for highbd frames.

BUG=aomedia:1246

Change-Id: I36ff4ddbb9b2e884e4a5b76485247a20b1f5db3c

BUG=aomedia:1232

Change-Id: Ib2578cdb41128997f7513bc11f86edf43a799959

This experiment has been abandonned for AV1.

Change-Id: Ief8ed6a51a5e7bac17838ebb7a88d88bbf90a96f

Record total rate cost when computing trellis optimization.
Reduce redundant rate computation in later stages.

Speed impact: ~6% speed up
Coding performance should not be affected.

Change-Id: I9e940a2d126bb55930fcf22ea04d061eee1fc944

Encoder: Prior to this patch, some chroma data was calculated and
later discarded when in monochrome mode. This patch ensures that
the chroma planes are left uninitialised and that chroma
calculations are not performed.

Decoder: Prior to this patch, some chroma calculations were still
being performed in monochrome mode (e.g. loop filtering). This
patch ensures that calculations are only performed on the y
plane, with the chroma planes being set to a constant.

Change-Id: I394c0c9fc50f884e76a65e6131bd6598b8b21b10

Change-Id: I4d201631e288ca0f5a0d13583707dff483eb5cac

The x0 parameter can now be chosen on the encoder side
independently of the dq offset.

Change-Id: I4a63bae6ae81d5a7f65c53efe1b1ecd713355b01

Change-Id: I861ccc09454606ce4172db02a52550f9b7a7800c

This fixes several MSVC warnings.

Change-Id: Id94f509a56d195afb48c3f2fa71986e0d8baafb4

Change-Id: I8092e3bbabbcf5efb5d0c659418fdd3c6ad90f2c

when trellis quantizaiton is disabled, the b_quantizer should be used.
existing code has a lot of hard coded flag not hooked up correctly with
high level trellis quantization enable/disable flag, which causes
fp_quantizer is used. this patch is intended to remove the hard coded
flags and correctly get them from the high level control.

Change-Id: I0142aace03586d07aedea138fce08d3e72d99c32

This issue isn't triggered currently though, because encode_block() is
always called with enable_optimize_b = 1.

Change-Id: I1ce5a35222cb6a647418bf7e4dc3189db5aa3854

Change-Id: I87b11ec892fe5048f858869a3b6e68d0fe78ea43

When a transform block is skipped, reset the txb_entropy_ctx value
to be 0. This clears the assertion failure cases in txk-sel.

Change-Id: I23816761d4383aa9877bc54b117f54d69e4949a9

Add speed feature that uses hash tables to
reuse previously found optimized coefficients
in av1_optimize_txb. This skips some expensive
optimize_txb calls.

Currently shows no significant quality
degredation or speed improvement, and set to off
by default. Requires hash_me, lv_map and
lv_map_multi. Adding to speed features required
changing AV1_COMMON *cm to AV1_COMP *cpi in a
chain of functions.

Variations that have been tried:
-varying the maximum eob on which the feature
activates: 16, 32, 64. 16 currently used. 64
has best hit rate but longer execution time.
-varying the data hashed and the length of hashes
(first hash is 16 bit and based on context data,
while second hash is 16 bit and based only on
pre-optimized qcoeff values.)
-softening the data used for the hashes: ideally
this would raise the number of hits, without
compromising quality too much.

Change-Id: I94f22be82f3a46637c0489d512f2e334a307575f

Change-Id: I9c013f1c7bd1ae5adf58ac884f884ea0c59a382c

This allows dropping the code for non-scaled (VP9) transforms,
who don't have highbitdepth implementations anymore anyway.

Change-Id: I00071e1ca05f642eeba071849bb85b9b8b61c50f

Properly store and update the luma component result in the key
frame rate distortion optimization process for chroma component
prediction. The txk-sel provides 0.15% gains for key frame coding
on top of lv-map/multi and all other default experiments.

Change-Id: I694860607b318d0a84333ed928d3de85c7676623

If the transform block selects a transform kernel type that makes
it all zero coefficients and hits the best rd cost, store such
decision and force the transform block to be coded as all zero
block in the final coding pass. This makes txk-sel improve the
kf coding performance by 0.15% now.

Change-Id: I4bc3c185b9c68daefb856276da45bbf9e317d02e

Change-Id: I523d9a89493895eb6a7af1df30a39f36ca9f159f

Bakes in DISABLE_VARTX_FOR_CHROMA/DISABLE_SMLTX_FOR_CHROMA_INTRA
macros.
Cleans up code and removes the unwieldy uv_txsize_lookup array.

Change-Id: I9aec8bfcfe218791517ab18e427bc645687916c3

The speed feature FINAL_PASS_TRELLIS_OPT is meant to disable
optimize_b during the transform search but allow it for the
final encode of blocks. There was a previously existing macro
called DISABLE_TRELLISQ_SEARCH, which does the same thing. This
patch merges the functionality so the macro serves only to enable
the speed feature.

Change-Id: Ieee70f97f817998b7ca275f6e4647cc89a330ad6

Removes existing inconsistencies between chroma tx_sizes
used for chroma loopfiltering.
Includes various refactoring to remove the uv_txsize_lookup
array eventually.

BUG=aomedia:1090

Change-Id: Ib74299b41280ca3ebeaf9a9293242d531d68ad28

Removing code for this option since it is not better than the
DISABLE_VARTX_FOR_CHROMA = 1 option and is more complex.

Change-Id: Id39d23bc6130bbed0ac008c1c76a2ba5aaee4d22

Change-Id: Ieb40253c3a0dcb81fc068fa41e23c867bddef22a

Deletng the code since it is in fact for the PVQ, not for dist-8x8.

The relavant code removal for PVQ is commit:
d0b77ac7
"Remove PVQ code" on 2017-10-20, which has removed the PVQ code.

Change-Id: I6b58d0acedb6bf278f35ac6c76a7a0a481cd3edb

Some refactoring done to enable reuse of code.

Change-Id: I28cea413a1bf7e6b8ce82ee50c5b5603e8185a2a

Change-Id: I6db5563ec4f4992a899090cf15c3f5f5125a0764

The get_y_mode function, is superfluous, not used consistently, and requires a
useless block_idx parameter than gets pass around a lot inside the
codebase for no apparent reason.

The block parameter is misleading, as it could cause people to think all these
functions actually use this value.

Change-Id: I7ae0a8d1282c009b9114c83771cce10f5c2ee397

Based on the HW Subgroup call of December 4th 2017, we limit luma partition to
32X32.

Regression on Subset 1
  PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
0.0881 |  1.3504 |  1.2936 |   0.0572 |  0.0182 |  0.0227 |     0.5204

https://two.arewecompressedyet.com/?job=CfL-PartU%402017-12-12T15%3A39%3A36.794Z&job=CfL-Max32x32%402017-12-12T16%3A10%3A09.989Z

Change-Id: I7e3cfd68097c0bc24b1426348b5fd574c4f638a0

To avoid a cascade of encodes when performing CfL RDO,
we compute DC_PRED on the partition unit. To do so, we
change the tx_size of CfL to match the size of the
partition unit (i.e. CfL partitions only contain 1
transform block).

This change requires disabling CfL when a chroma
partition-unit-sized DC_PRED is unavailable
(i.e. 4:1, 1:4 partitions and chroma partitions > 32X32).

Results on Subset1 (compared to disabling 4:1 and 1:4 partU):
   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.1243 | -1.9286 | -2.0140 |  -0.1514 | -0.1512 | -0.1947 |    -0.8066

https://two.arewecompressedyet.com/?job=master%402017-12-12T14%3A53%3A01.451Z&job=CfL-PartU%402017-12-12T15%3A39%3A36.794Z

Change-Id: I2a4adde79c10089130775b8e0df5f9c198855cad

Change-Id: I4eadddf7c3e99a9a71fd858e46970c1b7d3cd713

Change-Id: I2212609532783849ac5d8c91aec6e34af48f01f7

Now it supports inter blocks only

Change-Id: I6307664452588e654853b4ff18b83cce0343268b

Also, splits the sub_tx_size_map array into inter and intra
in order to enable the new 4:1 transforms for inter and
intra separately.
Includes refactoring such as removing the intra_tx_size_cat_lookup
array since it is unnecessary, and consolidating the
max_txsize_rect_lookup array for convenience.

Change-Id: I112553bab612dafb973611c87f36a43e1ac4be85

Rather than having a set of parameters for each of the 6
COEF_BANDS, we have 1 for DC and 1 for AC coefficients.
No change in performance since all of the bands had the
same parameters.

Change-Id: I3665e7c1b21f117be776f371d87d64b097715735

Change-Id: I09cf970a9be3a7dfe7e134bec5d778cda77f2682

This simplifies the profile lookup table and removes the requirement
of computing the ctx for new-quant. This change produces a
0.01% drop in performance on 40 frames of lowres.

Change-Id: Ifcbd7d6e427090e02bb43f6de15e551796aa74c4