GitLab

This commit allows the codec to analyze the reference motion vector
candidate list and adaptively reduce the size of inter prediction
mode set.

Change-Id: Ied6a403843b860d66f26ed485c1825c05c71bdfc

This commit re-works the entropy coding scheme of the motion
compensated prediction modes. It allows more flexible hyperplane
partition for precise classification.

Change-Id: Iba5035c76691946cf1386b6c495e399c3d9c8fc5

Change-Id: Ide48fa4312f7828f99290f7a2be878f5673fa716

It was removed by the master branch merge.

Change-Id: I4b2a524c9e052e41063359afcb4ba22bf78344cf

Store it in nextgenv2 for now.

Change-Id: Iab0af0e15246758e3b6e8bde4a74b13c410576fc

Estimate angle histogram using gradient analysis, then skip those
angles that are unlikely to be chosen.

On ext-intra experiment, turning off filter-intra modes:
for all-key-frame setting, computation overhead is reduced
by about 40%, coding gain dropped from +2.08% to +1.96% (derflr);
with kf-max-dist=150, computation overhead is reduced
by about 60%, coding gain dropped from +0.58% to +0.49% (derflr).

Change-Id: I36687410fb10561b8e1a8eebb1528cf17755bd5b

Take out an unnecessary use of the mode context array.

Change-Id: I4032ed18464e5ec49a2160bea964bad9b716ee54

Move txfm functions from encodemb to hybrid_twd_txfm.c
to make encodemb's code flow clear

Change-Id: If174d8ddb490d149c103e5127d30ef19adfbed13

This commit ports the motion vector stack from motion field
analyzer to the encoding and decoding pipeline.

Change-Id: Ie283c1e1a15b4c17a1c7c175ce322bf053bb7840

Under the experiment of EXT_REFS: LAST2_FRAME, LAST3_FRAME, and
LAST4_FRAME.

Coding efficiency: derflr +1.601%; hevchr +1.895%
Speed: Encoder slowed down by ~75%

Change-Id: Ifeee5f049c2c1f7cb29bc897622ef88897082ecf

Found with clang -fsanitize=integer

Change-Id: I2538e7483cb2d5f06bceecbd3326bdd88bfecfa1

Coding gain on derflr drops to +1.83%.

Change-Id: If68c429f09422a70513d9f1e8e36e10c928e034a

Remove MISC_FIXES flags except for the changes on MV precision, which
has a 0.1% performance drop.

On derflr, the impact is -0.012%.

Change-Id: I0a74e5a212dd0cb827192a318c92a714c9681e45

Change-Id: Ia7680ddf00dd50dd66bbb5753bae30b937988800

Reset the effective range of inter_tx_size, instead of the entire
array in the rate-distortion optimization loop.

Change-Id: Id453fbd6dddfe69f4e451ba8518c083326d5dd53

Coding gain remains about the same, while overall speed is
substantially increased.

Change-Id: I2989bebcfd21092cd6a02653d4df4a3bf6780874

This commit fixes an encoding failure case triggered when early
termination feature is turned on for transform block size search.
It resolves the corresponding enc/dec mismatch issue.

Change-Id: I2c5b7d8b1efe25fe3810e6ed307f4b1865dede49

Javan Whistling Duck release.

Change-Id: If44c9ca16a8188b68759325fbacc771365cb4af8

On derflr, +0.1% for VP10; however, -0.03% on VP9.

Change-Id: I09c724232ede74254043d61d3cadc506256af0af

Adds a new interpolation experiment.

Improves entropy coding to send the filter type only if
the motion vectors have subpel components.
Adds one new 8-tap smooth filter, and tweaks the others.

derflr: +0.695%
hevcmr: +0.305%

About 5% encode slowdown. No visible impact for decoding.

Also makes the interpolation framework flexible to support both
strictly interpolating filters as well as non-interpolating
filters that filter integer offsets. This is mainly for
further experimentation and if not found useful the code will
be removed.

Change-Id: I8db9cde56ca916be771fe54a130d608bf10786e6

This commit refactors the rate-distortion optimization scheme for
transform block coding. When both ext-tx and var-tx experiments
are turned on, the encoding time for bus_cif at 1000 kbps goes down
from 706377 ms to 666503 ms (5.6% speed-up). The coding statics
remain unchanged.

Change-Id: I20835db573725580aad79c16220f799ce01f2093

Currently there are two parts in this experiment: extra directional intra
prediction modes and the filter intra modes migrated from the nextgen branch.

Several macros are defined in "blockd.h" to provide controls of the experiment
settings. Setting "DR_ONLY" as 1 (default is 0) means we only use directional
modes, and skip the filter-intra modes; "EXT_INTRA_ANGLES" (default is 128)
defines the number of different angles we want to support; setting
"ANGLE_FAST_SEARCH" as 1 (default is 1) means we use fast sub-optimal search
for the best prediction angle, instead of exhaustive search. The fast search
is about 6 times faster than the exhaustive search, while preserving about
60% of the coding gains.

With extra directional prediction modes (fast search), we observe the following
code gains (number in parentheses is for all-key-frame setting):
derflr +0.42%  (+1.79%)
hevclr +0.78%  (+2.19%)
hevcmr +1.20%  (+3.49%)
stdhd  +0.56%
Speed-wise, about 110% slower for key frames, and 30% slower overall.

The gains of filter intra modes mostly add up with the gains of directional
modes. The overall coding gain of this experiment:
derflr +0.94%
hevclr +1.46%
hevcmr +1.94%
stdhd  +1.58%

Change-Id: Ida9ad00cdb33aff422d06eb42b4f4e5f25df8a2a

This commit re-works the rate-distortion optimization scheme for
transform coding. It improves the overall compression performance.
For derf set, the ext-tx experiment provides 2.27% coding gains,
and the new scheme that integrates multiple transform type selection
and recursive transform block partitioning provides a total of 3.24%
coding gains.

Change-Id: Ia1887c4c44b73dfb915d091d96660a99f09d5cc3

This commit hooks up the rate-distortion optimization system to
fully exploit recursive transform block partition and multiple
transform type. The compression performance of the two experiments
largely adds up. For derf set, ext-tx provides additional 2.1%
coding gains on top of the gains due to recursive transform block
partition (0.69%).

Change-Id: I1091fb9545f74e489a6a2489dc3c12f5abd05043

This commit allows the loop filter to account for the recursive
transform block partition when selecting the filter and mask.

Change-Id: I62b6c2dcc0497cbe1f264b03c46163f55d2c9752

Change-Id: I3d90a0286c5ef559b91ad298db97e8990becf85f

Allow the encoder to drop individual transform block coding.

Change-Id: I2c2b2985254cb92baf891f03daa33f067279373b

This commit enables the entropy coding of the recursive transform
block partition syntax.

Change-Id: I0c2509fb7b9822d12a721f9ebf9327fac83c777e

Allows inter and intra tx_types to have different sets of
transforms for different tx_size/sb_type combinations.

Change-Id: Ic0ac1daef7a9fb15c4210271e4d04cd36e5cec8e

Rework the rate distortion optimization pipeline. Use precise
distortion metric that accounts for the forward and inverse
transform rounding effect.

Change-Id: Ibe19ce9791ec3547739294cc3012dd9e11f4ea49

This commit makes the coefficient token packtization process account
for variable transform block sizes supported in a single processing
block. It fixes an enc/dec mismatch issue when var-tx, ext-tx, and
misc-fixes experiments are all turned on.

Change-Id: I2e8946e6f72de567603a568debbadad11196430c

Properly reset the early termination flag in the recursive transform
block partitioning rate-distortion optimization scheme.

Change-Id: Ibfe918f21f11dcb1ec267c09f954c635305cc95a

Use inter_block_yrd as rate-distortion optimization for lossless
coding. This fixes transform coefficient buffer swap use case and
resolves the unit test failure related to lossless coding.

Change-Id: I1512dab5ed5760c31f7de21a06e8d9ed1eb081fa

This commit makes the recursive transform block partitioning properly
handle the non-420 color format. It resolves an enc/dec mismatch
issue in that setting when var-tx experiment is turned on.

Change-Id: I48a91de02c11b3153f897d1cca0ae948eec15605

Add the row and column index to the argument list of unit functions
called by foreach_transformed_block wrapper. This avoids the
repeated internal parsing according to the block index.

Change-Id: Ie7508acdac0b498487564639bc5cc6378a8a0df7

This commit fixes the merge conflicts between master and nextgenv2 and
disable early termination in choose_tx_size() to avoid failure in test.

The test failures are pre-existing, some of the issue were fixed in
masterbase already, so will have another merge to introduce the fixes.

Change-Id: Ib71889661955e73aedbb4db49d8be70425281dcb

Temporarily reset the transform type in the inter modes when
recursive transform block partitioning is used. This resolves an
enc/dec mismatch issue in nextgenv2 codebase when both var-tx and
ext-tx experiments are turned on.

Change-Id: I2543f0a567243da95b237752d46964b07b669ad9

Clear the compiler errors when both high bit-depth and recursive
transform block partition experiments are enabled.

Change-Id: If0b6396851f10c28b4f26350322ccd1ba2fc9aff

Some more testing of this patch would probably be useful, but I
think the basics of it should work fine now.

See issue 1035.

Change-Id: I4a36d58f671c5391cb09d564581784a00ed26245

This experiment allows using full above/right edges for all transform
sizes whenever available (for d45/d63), and adds bottom/left edges for
d207.

See issue 1043.

Change-Id: I5cf7f345e783e8539bb6b6d2c9972fb1d6d0a78b