GitLab

This fixes several MSVC warnings.

Change-Id: Id94f509a56d195afb48c3f2fa71986e0d8baafb4

Change-Id: I8092e3bbabbcf5efb5d0c659418fdd3c6ad90f2c

when trellis quantizaiton is disabled, the b_quantizer should be used.
existing code has a lot of hard coded flag not hooked up correctly with
high level trellis quantization enable/disable flag, which causes
fp_quantizer is used. this patch is intended to remove the hard coded
flags and correctly get them from the high level control.

Change-Id: I0142aace03586d07aedea138fce08d3e72d99c32

This issue isn't triggered currently though, because encode_block() is
always called with enable_optimize_b = 1.

Change-Id: I1ce5a35222cb6a647418bf7e4dc3189db5aa3854

Change-Id: I87b11ec892fe5048f858869a3b6e68d0fe78ea43

When a transform block is skipped, reset the txb_entropy_ctx value
to be 0. This clears the assertion failure cases in txk-sel.

Change-Id: I23816761d4383aa9877bc54b117f54d69e4949a9

Add speed feature that uses hash tables to
reuse previously found optimized coefficients
in av1_optimize_txb. This skips some expensive
optimize_txb calls.

Currently shows no significant quality
degredation or speed improvement, and set to off
by default. Requires hash_me, lv_map and
lv_map_multi. Adding to speed features required
changing AV1_COMMON *cm to AV1_COMP *cpi in a
chain of functions.

Variations that have been tried:
-varying the maximum eob on which the feature
activates: 16, 32, 64. 16 currently used. 64
has best hit rate but longer execution time.
-varying the data hashed and the length of hashes
(first hash is 16 bit and based on context data,
while second hash is 16 bit and based only on
pre-optimized qcoeff values.)
-softening the data used for the hashes: ideally
this would raise the number of hits, without
compromising quality too much.

Change-Id: I94f22be82f3a46637c0489d512f2e334a307575f

Change-Id: I9c013f1c7bd1ae5adf58ac884f884ea0c59a382c

This allows dropping the code for non-scaled (VP9) transforms,
who don't have highbitdepth implementations anymore anyway.

Change-Id: I00071e1ca05f642eeba071849bb85b9b8b61c50f

Properly store and update the luma component result in the key
frame rate distortion optimization process for chroma component
prediction. The txk-sel provides 0.15% gains for key frame coding
on top of lv-map/multi and all other default experiments.

Change-Id: I694860607b318d0a84333ed928d3de85c7676623

If the transform block selects a transform kernel type that makes
it all zero coefficients and hits the best rd cost, store such
decision and force the transform block to be coded as all zero
block in the final coding pass. This makes txk-sel improve the
kf coding performance by 0.15% now.

Change-Id: I4bc3c185b9c68daefb856276da45bbf9e317d02e

Change-Id: I523d9a89493895eb6a7af1df30a39f36ca9f159f

Bakes in DISABLE_VARTX_FOR_CHROMA/DISABLE_SMLTX_FOR_CHROMA_INTRA
macros.
Cleans up code and removes the unwieldy uv_txsize_lookup array.

Change-Id: I9aec8bfcfe218791517ab18e427bc645687916c3

The speed feature FINAL_PASS_TRELLIS_OPT is meant to disable
optimize_b during the transform search but allow it for the
final encode of blocks. There was a previously existing macro
called DISABLE_TRELLISQ_SEARCH, which does the same thing. This
patch merges the functionality so the macro serves only to enable
the speed feature.

Change-Id: Ieee70f97f817998b7ca275f6e4647cc89a330ad6

Removes existing inconsistencies between chroma tx_sizes
used for chroma loopfiltering.
Includes various refactoring to remove the uv_txsize_lookup
array eventually.

BUG=aomedia:1090

Change-Id: Ib74299b41280ca3ebeaf9a9293242d531d68ad28

Removing code for this option since it is not better than the
DISABLE_VARTX_FOR_CHROMA = 1 option and is more complex.

Change-Id: Id39d23bc6130bbed0ac008c1c76a2ba5aaee4d22

Change-Id: Ieb40253c3a0dcb81fc068fa41e23c867bddef22a

Deletng the code since it is in fact for the PVQ, not for dist-8x8.

The relavant code removal for PVQ is commit:
d0b77ac7
"Remove PVQ code" on 2017-10-20, which has removed the PVQ code.

Change-Id: I6b58d0acedb6bf278f35ac6c76a7a0a481cd3edb

Some refactoring done to enable reuse of code.

Change-Id: I28cea413a1bf7e6b8ce82ee50c5b5603e8185a2a

Change-Id: I6db5563ec4f4992a899090cf15c3f5f5125a0764

The get_y_mode function, is superfluous, not used consistently, and requires a
useless block_idx parameter than gets pass around a lot inside the
codebase for no apparent reason.

The block parameter is misleading, as it could cause people to think all these
functions actually use this value.

Change-Id: I7ae0a8d1282c009b9114c83771cce10f5c2ee397

Based on the HW Subgroup call of December 4th 2017, we limit luma partition to
32X32.

Regression on Subset 1
  PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
0.0881 |  1.3504 |  1.2936 |   0.0572 |  0.0182 |  0.0227 |     0.5204

https://two.arewecompressedyet.com/?job=CfL-PartU%402017-12-12T15%3A39%3A36.794Z&job=CfL-Max32x32%402017-12-12T16%3A10%3A09.989Z

Change-Id: I7e3cfd68097c0bc24b1426348b5fd574c4f638a0

To avoid a cascade of encodes when performing CfL RDO,
we compute DC_PRED on the partition unit. To do so, we
change the tx_size of CfL to match the size of the
partition unit (i.e. CfL partitions only contain 1
transform block).

This change requires disabling CfL when a chroma
partition-unit-sized DC_PRED is unavailable
(i.e. 4:1, 1:4 partitions and chroma partitions > 32X32).

Results on Subset1 (compared to disabling 4:1 and 1:4 partU):
   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.1243 | -1.9286 | -2.0140 |  -0.1514 | -0.1512 | -0.1947 |    -0.8066

https://two.arewecompressedyet.com/?job=master%402017-12-12T14%3A53%3A01.451Z&job=CfL-PartU%402017-12-12T15%3A39%3A36.794Z

Change-Id: I2a4adde79c10089130775b8e0df5f9c198855cad

Change-Id: I4eadddf7c3e99a9a71fd858e46970c1b7d3cd713

Change-Id: I2212609532783849ac5d8c91aec6e34af48f01f7

Now it supports inter blocks only

Change-Id: I6307664452588e654853b4ff18b83cce0343268b

Also, splits the sub_tx_size_map array into inter and intra
in order to enable the new 4:1 transforms for inter and
intra separately.
Includes refactoring such as removing the intra_tx_size_cat_lookup
array since it is unnecessary, and consolidating the
max_txsize_rect_lookup array for convenience.

Change-Id: I112553bab612dafb973611c87f36a43e1ac4be85

Rather than having a set of parameters for each of the 6
COEF_BANDS, we have 1 for DC and 1 for AC coefficients.
No change in performance since all of the bands had the
same parameters.

Change-Id: I3665e7c1b21f117be776f371d87d64b097715735

Change-Id: I09cf970a9be3a7dfe7e134bec5d778cda77f2682

This simplifies the profile lookup table and removes the requirement
of computing the ctx for new-quant. This change produces a
0.01% drop in performance on 40 frames of lowres.

Change-Id: Ifcbd7d6e427090e02bb43f6de15e551796aa74c4

BUG=aomedia:1065

Change-Id: I35eda84e249fc42e319604079c122df8ab101f90

16x64 reuses scan order of 16x32
64x16 reuses scan order of 32x16

max eob is curtailed to 512 (instead of 1024) for both.

Change-Id: Iac2145aa5e3d090009e2a2f5715caa8d84dfb2ee

Change-Id: Iff2780944e954973b9ee9ab3c04d1e6b44a7810d

The CfL context is now stored inside MACROBLOCKD instead of
MACROBLOCKD only storing a pointer to the CfL context.
The intent is to avoid race conditions as MACROBLOCKD is stored
inside ThreadData. This change also simplifies CfL Initialization.

Change-Id: I991503716b21fc9aca60caddb2008b8bff397e6d

Change-Id: I4b1a386ff989d63c7230923b36cb230b18466143

The change makes the entropy context for transforms use
the same mechanism as with and without lv_map.
For the non-lv-map case the context is now based on the
the larger transform dim for 2:1 rect transforms. The context
is now the average for 4:1 rect transforms for both lv-map and
non-lv-map cases.
There is one small fix for level map for getting the correct
rate when skip is set.

BDRATE: lowres, 30 frames, speed 1: -0.15% gain for the
non-lv-map case on the baseline.

Change-Id: I06a583d33bef68202d72a88e077f8d31cc5e7fe4

This experiment has been abandonned for AV1.

Change-Id: Ib9a8eea6b0ad5580dc81836be8fc5e898bb9bb45

Adds an option to split chroma tx size by one level if the luma
transform size is split at least one level. Otherwise use the
largest transform size for chroma.
This option is enabled with DISABLE_VARTX_FOR_CHROMA set as 2,
but is currently turned off until we finish testing.

Change-Id: Ie048c4e33193e0cc7e1bbee37e4ccc085834433c

This mistake was fixed in a8db0e8d
and brought back in for
the daala-tx case in 4a05a58c.

Change-Id: I9cf36b63f2dacd9d01887bfaacd8b318245f882a

When the flag DISABLE_VARTX_FOR_CHROMA is on chroma is
constrained to always use the largest transform size
for the prediction unit size.
This is meant to simnplify the logic for transform size
selection for chroma with hopefully no loss.

Results:
lowres 30 frames, speed 1: -0.038% (a slight improvement).
lowres 30 frames, speed 0: 0.000% (noise level difference).

Change-Id: I14dd5b1983d908bd98e59b7d252e11f5755c97e6