GitLab

changed the token cost for 8x8 transformed macroblock used in trellisquant
from those derived from 4x4 transform coefficient distribution to those
derived from 8x8 transform coefficient distribution. Test results show
this fix help 8x8 transform based compression consistently on cif and hd
sets:

http://www.corp.google.com/~yaowu/no_crawl/t8x8/cif_cost8x8only.html
(avg psnr:.14% glb psnr: .17% ssim: .20%)
http://www.corp.google.com/~yaowu/no_crawl/t8x8/hd_cost8x8only.html
(avg psnr:.17% glb psnr: .18% ssim: .58%)

Note: To test the effect of this change, 8x8 transform was forced to be used
only on 16x16 predicted macroblocks on inter frames, the effect would be
bigger had all macroblocks been forcd to use 8x8 transform.

Change-Id: If9b7868b75357c66541f511e5ee78e4d2d4929a4

This commit added a set of loop filter functions for macroblocks
using 8x8 transform. First we turned off the regular loop filtering
on 4x4 block boundaries that do not exist in macroblocks using 8x8
transform. Second, we change to use the same loop filter(mask and
7 tap filter) that used for macroblock edge filtering.

Change-Id: I3a00460b7674ced116917d86812ffc32578c1d3a

This fixed a conflict introduced by the change of adding 8x8 intra
prediction modes. The 8x8 intra prediction mode code assumed the
use of 4x4 transform, and causes encoder crashes when the codec is
configured with --enable-t8x8.

Change-Id: I00cc94df63e9725377ffba9eb51be6b77fe3fcf9

commit cf561bad accidentally deleted a line of code that sets the
base_qindex for each frame, which leads to every frame is encoded
at Q of 0.

Change-Id: Ib5f8022e856bf3b3bd0d4147405e46241e3dcf2d

The commit adds a new set of loop filter for macroblock edge filtering.
The new loop filter has a mask to detect so-called "flat" regions. The
detection checks 5 pixels of each side of an edge. If the all pixels
have value with +/-1 from the edge pixel on the same side, the region
is treated as a "flat" region. For such case, a 7 tap filter is used
to change 3 pixel values on each side. The 7 taps are:
               [1, 1, 1, 2, 1, 1, 1]/8
The furthest away pixels used as input are +/-5 away from edge. For
non-flat region, we fall back to old filtering. It should be noted
here that the thresholds and filter taps may require more optimization
for best possible results.

Tests on a set of hd clips showed consistent gains:
http://www.corp.google.com/~yaowu/no_crawl/mblpf_hd.html
(avg psnr: .83% glb psnr: .77% ssim: .82%)

Tests on derf set also showed consistent gains:
http://www.corp.google.com/~yaowu/no_crawl/mblpf_derf.html
(avg psnr: .24% glb psnr: .22% ssim: .48%)

Change-Id: I0855b1ff48e79e1175c20b81967137e18b2af352

Put traps to prevent two possible divide by 0 errors.

Change-Id: Ia415b945244253dcdd12f54f1f157f9ca8c94d6b

A problem can arise on static clips with force key frames where
attempts to avoid popping lead to a progressive reduction in key
frame Q that ultimately may lead to unexpected overspend against
the  rate target.

The changes in this patch help to insure that in such clips the
quality of the key frames across the clip is more uniform (rather
than starting bad and getting better - especially at low target rates).

This patch also includes a fix that removes a delta on the Y2DC
when the baseline q index < 4 as this is no longer needed.

There is also a fix to try and prevent repeat single step Q adjustment in
the recode loop leading to lots of recodes, especially where the use
of forced skips as part of segmentation has made the impact of Q on
the number of bits generated much smaller.

Patch 2: Amend "last_boosted_qindex" calculation for arf overlay frames.

Change-Id: Ia1feeb79ed8ed014e4239994fcf5e58e68fd9459

In certain hardware configuration, where mmx code is enabled and
other simd (sse2/sse3) disabled, lacking of this emms caused invalid
internal stats outputs.

Change-Id: I77c61cf6e0448d3f3b8c11781aa9e42f31d231c9

Change-Id: Ifadf65026a11bdb5d39840748613880bcfb364bb

Change-Id: I6bbed8bcb2dfa3458ffc59179dfba66c92e18125

Previously, the mode context is always udpated based on stats of current
frame, when there is no count, 50% is used for both left and right branch.
However, it is observed that with such strategy, a small count or no count
at all can skew the probability distribution significantly. This commmit
changed the mode_context update strategy to prevent small counts from
skewing the probability distributions.

Tests on derf set showed a small gain:  .06% in psnr and .09% in ssim

Change-Id: Ic812e64ae5f70251c170b0717f7b7fa587055488

Cleanup and switch to Q extended at low end too.

Change-Id: Ie22676bb9e961097d75dbd1d81745208b63e5f4b

This commit extends the number of Q steps to 256 from 128.
The q_trans[] array has been altered to distribute available Q index values
(using the current 64 steps available as input parameters) evenly across the
available range. This is coupled with the fact that each Q step where possible
now equates to a fixed % change in the quantizer. This may want refinement
later especially in terms of the granularity at the high quality end but is a
reasonable starting point.

Change-Id: I2aaa6874fa10ce05c958dd182947ce39f6f1eecb

High Q end extended a little.
Some clean up.

Slightly better on SSIM, Slightly worse on PSNR over derf set.

Change-Id: I3dceea8a39e11c26e1a389a40e40b86efc76d28c

Added code to support 256 index steps instead of 128 but disabled for now.
Replace hard wired table vp8cx_base_skip_false_prob[128]
Observed Qindex problem with setting minimum loop filter value.
(Experiment code using real Q in place but for now just returning 0. This has a big
beneficial effect on some clips, particularly waterfall which shows 5% ssim gain)

Change-Id: I2f7117de8adc1797164c106aa13effc900a1467e

Whilst the encoder explicitly set the segment_id to 0
when segmentation is diabled, the decoder would allow
the segment_id to persist from the previous frame.

This fix attempts to make the decoder behave the same
as the encoder by explicitly setting the segment_id to
0 in this case.

Change-Id: I65c3a05247550edb10706eb5d54d306dfb792309

mode_info_context is padded with an additional column of data, so
mode_info_stride should be used to move between rows rather than
mb_cols.

Change-Id: I598559a2cd9df1c486d64aaeccf76b76a7ecf21c

Both encoder & decoder were using mb_cols to
offset from one row of MODE_INFO structures to the next
when they should have been using mode_info_stride.

Fixing this in both encoder and decoder gives around
a 3KB size saving and 0.025dB PSNR improvement on the one
720P clip I tried.

(Also removed "index" which was being updated but not used)

Change-Id: I413bea802b142886bfcf8d8aa7f5a2f0c524fd4b

Removed a couple more fixed tables for the extended quantizer experiment
that depend on QINDEX_RANGE.

Change-Id: I2c15ffc7488c2a2b8d6504e2c4b6b2339799d117

Previously, Y-adaptive UV intra coding only enabled on key frames in
UVINTRA experiment. This commit enabled the same coding for inter
frames, so the encoding of UV intra modes are consistent cross all
frame types. Tests on derf set showed a very small overall gain around
.04%:

http://www.corp.google.com/~yaowu/no_crawl/interUVintra.html

The gain looks to be reasonable given inta coded MBs is only a
small portion of MBs in inter frames.

Change-Id: Ic6fc261923f2c253f4a0c9f8bccf4797557b9e16

update_mbgraph_frame_stats used xd->mode_info_context
before it had been setup, resulting in potentially
random accesses of uninitialized memory.

This fix allocates a local MODE_INFO structure to hold
the data generated in the function.

Change-Id: Ic9e75610008ce0e2d690e8e583c21582fee6fc45

A previous commit 76feb965 made the vp8_mode_context adaptive on a frame
frame basis, this commit further made the coding context adaptive to two
frame types separately. Tests on derf set showed a further small gain on
all metrics: avg psnr 0.10%, glb psnr: 0.11%, ssim: 0.08%

http://www.corp.google.com/~yaowu/no_crawl/newNearMode_1209.html

Change-Id: I7b3e32ec8729de1903d14a3f1213f1624b78cdee

Experiment with old Q range but new higher precision quantizer
and transform code.

Change-Id: Id1ff4cb433e5775d709d0133e2aec0322975c292

The commit fixed a problem by capping cpi->active_best_quality to be
smaller than cpi->worst_quality.  Also fixed a few line of code that
was misplaced.

Change-Id: Ie908264b72140c669122a0afde5d886619c33474

This commit removed the macro CONFIG_MULCONTEXT, which was used to
indicate the experiment code for using separate context for altref
and normal frames. This commit made the change fully merged in.

Change-Id: I525f927f68e2365d37b340ef23b836a136a4f70b

This commit removed the macro CONFIG_I8X8, which was used to indicate
the 8x8 intra prediction experiment, made the change fully merged in.

Change-Id: Iafa4443781ce6e83f5591c12ba615a0e92ce0ea0

vp8_mode_contexts[] is an entropy table used to code inter mode
choices. It was a fixed constant table. This commit made the entropy
context adaptive. Tests on derf set showed very good consistent gains
on all metrics: avg psnr .47%, overall psnr .46% and ssim .40%.

http://www.corp.google.com/~yaowu/no_crawl/newModeContext.html

Change-Id: Ia62b14485c948e2b74586118619c5eb2068b43b2

The MODE_STATS macro was used to #ifdef around code for mode entropy
stats collection, this commit fixed a crash when MODE_STATS is on.
The commit also changed a number of array definitions to use defined
macros instead of hard-coded numbers.

Change-Id: I114592f53a1e44e31e455f5725f036ae6168735a

fixed issues caused by conflicts between two experiments.

Change-Id: I56a9bd69493e4850c121ea057a6233c55777c2a5

Because the variable doesn't distinguish between DC and non-DC
prediction, but rather between 16x16 or 4x4 prediction.

Change-ID: Ia4e7dda2bd6230c91515072e3277be2d64e42629