Incorporates mv_ref, mbsplit and second_mv into the adaptive
entropy framework. The mv_ref framework has been modified from
before.

Adds some clean-ups and fixes.

Results with the adaptive entropy experiment are currently up by
+1.93% on derf; +2.33% std-hd and +1.87% yt-hd.

Fixed a nasty intermittent bug.

Change-Id: I4b1ac9f9483b48432597595195bfec05f31d1e39

This patch incorporates adaptive entropy coding of coefficient tokens,
and mode/mv information based on distributions encountered in a frame.
Specifically, there is an initial forward update to the probabilities
in the bitstream as before for coding the symbols in the frame, however
at the end of decoding each frame, the forward update to the
probabilities is reverted and instead the probabilities are updated
towards the actual distributions encountered within the frame.
The amount of update is weighted by the number of hits within each
context.

Results on derf/hd/std-hd are all up by 1.6%.

On derf, the most of the gains come from coefficients, however for the
hd and std-hd sets, the most of the gains come from the mode/mv
information updates.

Change-Id: I708c0e11fdacafee04940fe7ae159ba6844005fd

This commit merge the QI mode experiment. As the experiment affects
the encoding of intra coding modes on key frame only, the overall
effect of the experiment on encoding tests is insignificant.

Change-Id: I9e4e3933adface88867ad429cee3986e529c511d

The commit merges the UVINTRA experiment and removed the related
macros. The overall effect of the experiment is a small gain (.1%
on derf)

Change-Id: Ia34b3312fb9b5b34c9ba111bf0fa78c6f78ac80b

Change-Id: I0af3395500b1cb0ed629249eb6636a0c9322cb18

With the NEWENTROPY experiment enabled encoding certain clips
produced invlid bitstreams, or files that had a high degree
of artefacts.

This was the results of pointers in MACROBLOCKD not being
setup correctly (mode_info_context and prev_mode_info_context).

Change-Id: Ice13e1efa8bd122997d2f8f3f1e761c6c16e0403

This is the first patch to add superblock (32x32) coding
order capabilities. It does not yet do any mode selection
at the SB level, that will follow in a further patch.

This patch encodes rows of SBs rather than
MBs, each SB contains 2x2 MBs.

Two intra prediction modes have been disabled since they
require reconstructed data for the above-right MB which
may not have been encoded yet (e.g. for the bottom right
MB in each SB).

Results on the one test clip I have tried (720p GIPS clip)
suggest that it is somewhere around 0.2dB worse than the
baseline version, so there may be bugs.

It has been tested with no experiments enabled and with
the following 3 experiments enabled:
  --enable-enhanced_interp
  --enable-high_precision_mv
  --enable-sixteenth_subpel_uv
in each case the decode buffer matches the recon buffer
(using "cmp" to compare the dumped/decoded frames).
Note: Testing these experiments individually created
errors.

Some problems were found with other experiments but it
is unclear what state these experiments are in:
  --enable-comp_intra_pred
  --enable-newentropy
  --enable-uvintra

This code has not been extensively tested yet, so there
is every likelihood that further bugs remain. I also
intend to do some code cleanup & refactoring in tandem
with the next patch that adds the 32x32 modes.

Change-Id: I1eba7f740a70b3510df58db53464535ef881b4d9

Using contextual coding of the mkb_skip_coeff flag using the
values of this flag from the left and above. There is a small
improvement of about 0.15% on Derf:
http://www.corp.google.com/~debargha/vp8_results/mbskipcontext.html

Refactored to use pred_common.c by adding a new context type.

Results on HD set (about 0.66% improvement):
http://www.corp.google.com/~debargha/vp8_results/mbskipcontext_hd.html

Incliding missing refactoring to use the pred_common utilities.

Change-Id: I95373382d429b5a59610d77f69a0fea2be628278

The commit added a clamp to the 2nd motion vector used in compound
prediction to insure mv within UMV borders. The clamp is similar to
that of the first motion vector except that No SPLITMV is ever used
for the 2nd motion vector.

Change-Id: I26dd63c304bd66b2e03a083749cc98c641667116

Pulled out super block code for the snapshot as this
is not quite ready and will need an extensive re-merge.

Change-Id: I436369b511257447a7b0ea064016cb63f5011849

This gives a modest gain on derf overall, although at low bitrates the
cost is still too high, so this can be improved further.

Patch 2. Re-base and fix 80 column issues

Change-Id: Ida2f9fa3fe75370669f6a27b37108dc602231c63

The commit changed to compute UV intra RD estimates for 4x4 and 8x8
separately to be used in mode decision for MB modes associated with
the appropriate transform size respectively. Now finally after many
other changes related 8x8 quantizer zbin boost and zbin_mode_boost,
this change overall helps the HD(with 8x8) by around ~.13%.
(avg .13%  glb .13% ssim .17%)

The commit also has a few changes for eliminating compiler warnings.

Change-Id: Ibab35dad44820c87e6b44799c66f8d519cc37344

Change-Id: Ibcd2b9b247ff9f83331dac47f91ec285e8955ff1

Also remove duplicate build_intra_predictors_mby/uv().

Change-Id: I78607e7304952a9b962a5b25af9bb9c48692187b

This is the first patch for refactoring of the code related to
high-precision mv, so that 1/4 and 1/8 pel motion vectors can
co-exist in the same bit-stream by use of a frame level flag.
The current patch works fine for only use of 1/4th and
only use of 1/8th pel mv, but there are some issues with the
mode switching in between. Subsequent patches on this change Id
will fix the remaining issues.

Patch 2: Adds fixes to make sure that multiple mv precisions can
co-exist in the bit-stream. Frame level switching has been tested
to work correctly.

Patch 3: Fixes lines exceeding 80 char

Patch 4:
http://www.corp.google.com/~debargha/vp8_results/enhinterp.html

Results on derf after ssse3 bugfix, compared to everything
enabled but the 8-tap, 1/8-subpel and 1/16-subpel uv. Overall the
gains are about 3% now. Hopefully there are no more bugs lingering.
Apparently the sse3 bug affected the quartel subpel results more than
the eighth pel ones (which is understandabale because one bad predictor
due to the bug, matters less if there are a lot more subpel options
available as in the 1/8 subpel case).
The results in the 4th column correspond to the current settings.
The first two columns correspond to two settings of adaptive switching
of the 1/4 or 1/8 subpel mode based on initial Q estimate. These
do not work as good as just using 1/8 all the time yet.

Change-Id: I3ef392ad338329f4d68a85257a49f2b14f3af472

This is the initial patch for supporting 1/8th pel
motion. Currently if we configure with enable-high-precision-mv,
all motion vectors would default to 1/8 pel. Encode and
decode syncs fine with the current code. In the next phase
the code will be refactored so that we can choose the 1/8
pel mode adaptively at a frame/segment/mb level.

Derf results:
http://www.corp.google.com/~debargha/vp8_results/enhinterp_hpmv.html
(about 0.83% better than 8-tap interpoaltion)

Patch 3: Rebased. Also adding 1/16th pel interpolation for U and V

Patch 4: HD results.
http://www.corp.google.com/~debargha/vp8_results/enhinterp_hd_hpmv.html
Seems impressive (unless I am doing something wrong).

Patch 5: Added mmx/sse for bilateral filtering, as well as enforced
use of c-versions of subpel filters with 8-taps and 1/16th pel;
Also redesigned the 8-tap filters to reduce the cut-off in order to
introduce a denoising effect. There is a new configure option
sixteenth-subpel-uv which will use 1/16 th pel interpolation for
uv, if the motion vectors have 1/8 pel accuracy.

With the fixes the results are promising on the derf set. The enhanced
interpolation option with 8-taps alone gives 3% improvement over thei
derf set:
http://www.corp.google.com/~debargha/vp8_results/enhinterpn.html

Results on high precision mv and on the hd set are to follow.

Patch 6: Adding a missing condition for CONFIG_SIXTEENTH_SUBPEL_UV in
vp8/common/x86/x86_systemdependent.c

Patch 7: Cleaning up various debug messages.

Patch 8: Merge conflict

Change-Id: I5b1d844457aefd7414a9e4e0e06c6ed38fd8cc04

Remove error concealment code.

Change-Id: I882705174fbfea212e96f7f684e47a671dbe5c67

Change-Id: Ieaaa07c50eae41118596197f6a4d848135946e41

This commit only involves the removal of placeholder comments
//#if CONFIG_SEGFEATURES.

Change-Id: I94b350daaf998ee0cfdde5aa25b1d3b0522ab816

Merged in most of the current common prediction changes
that were under the #if CONFIG_COMPRED option.

Change-Id: If4e6f61dbe7b86dd449f6effbe93b5eb7e893885

Further changes to make experiments with the context
used for coding the dual pred flag easier.

Current best performing method tested on derf is a two
element context based on reference frame. I also tried
various combinations of mode and reference frame as
shown in commented out case using up to 6 contexts.

Derf +0.26 overall psnr +0.15% ssim vs original method.

Change-Id: I64c21ddec0abbb27feaaeaa1da2e9f164ebaca03

This commit merges the NEWNEAR experiment such that it
is effectively always on.

The fact that there were changes in the threading code again
highlights the need to strip out such features during the
bitstream development phase as trying to maintain this code
(especially as it is not being tested) slows the development cycle.

Change-Id: I8b34950a1333231ced9928aa11cd6d6459984b65

Initial modifications to make limited use of common prediction
functions.

The only functional change thus far is that updates to the probabilities are
no longer "damped". This was a testing convenience but in fact seems to
help by a little over 0.1% over the derf set.

Change-Id: I8b82907d9d6b6a4a075728b60b31ce93392a5f2e

Moved the prob_dualpred[] sturcture to common.
Created common prediction entry for Dual flag.

Change-Id: I9ac3d128bae6114f09e5c18216d4b95cf36453d5

Trial of a modified prediction function that ranks each possible
reference frame based on a combination of local usage and
frame level probability. The code is a bit cleaner and simpler.

In direct comparison with old unpredicted method with segment level
coding turned off for mode,ref & EOB the prediction gives a gain on derf
of around 0.4%. There is some further gain from bug fixes over earlier code.

With segment coding on the prediction method is slightly -ve on some very
easy clips (at low rates) due to slightly higher overheads, but better on harder
clips. Overall neutral on derf in direct comparison on latest code base, but
compared to earlier code without bug fixes about +0.7% overall psnr
+0.3% SSIM.

Change-Id: I5b8474658b208134d352d24f6517f25795490789

Extended prediction and coding of reference frame where
a subset of options are flagged as available at the segment level.

Updated copyright notices.

Switch to SAD in mbgraph code as SATD problematic for the
foreground and background separation as it can ignore large DC shifts.

Change-Id: I661dbbb2f94f3ec0f96bb928c1655e5e415a7de1

As a precursor to encoding 32x32 blocks this cl adds the
ability to encode the frame superblock (=32x32 block) at
a time. Within a SB the 4 indiviual MBs are encoded in
raster-order (NW,NE,SW,SE).

This functionality is added as an experiment which can be
enabled by ispecifying --enable-superblocks in the
command line specified to configure (CONFIG_SUPERBLOCKS
macro in the code).

To make this work I had to disable the two intra
prediction modes that use data from the top-right of the
MB.

On the tests that I have run the results produce
almost exactly the same PSNRs & SSIMs with a very
slightly higher average data rate (and slightly higher
data rate than just disabling the two intra modes in
the original code).

NOTE: This will also break the multi-threaded code.

This replaces the abandoned change:
Iebebe0d1a50ce8c15c79862c537b765a2f67e162

Change-Id: I1bc1a00f236abc1a373c7210d756e25f970fcad8

The bug was introduced by the commit that added I8X8 intra prediction
mode for inter frames, the decoder was not update to accept the additional
probability update from encoder. This causes the decoder typicall to crash
when encoder sends intra mode probability update.

Change-Id: Ib7dc42dc77a51178aa9ece41e081829818a25016

Please see the following for details:
https://gerrit.chromium.org/gerrit/#change,10925

Change-Id: Ie692261c255c58d7762df22eeca566a7d13adcba

This check in uses the common prediction interface functions
to code reference frame.

Some updates made regarding the impact of the new code in rd loop
but there remain TODOs in this regard.

Change-Id: I9da3ed5dfdaa489e0903ab33258b0767a585567f

This does not change any functionality just modifies the code to
use the common prediction module interface for coding
the segment data.

Change-Id: Ifd43e9153573365619774a4f5572215e44fb5aa3

In this commit only the decoder side was updated.

Change-Id: Ia9bd58da07d1a943f028e330f0489344e62b0d02

Moved some segmentation data structures into VP8_COMMON

Change-Id: I59c6e2edf7a0176e35319936eea450027aeb3b39

Whilst the encoder explicitly set the segment_id to 0
when segmentation is diabled, the decoder would allow
the segment_id to persist from the previous frame.

This fix attempts to make the decoder behave the same
as the encoder by explicitly setting the segment_id to
0 in this case.

Change-Id: I65c3a05247550edb10706eb5d54d306dfb792309

Both encoder & decoder were using mb_cols to
offset from one row of MODE_INFO structures to the next
when they should have been using mode_info_stride.

Fixing this in both encoder and decoder gives around
a 3KB size saving and 0.025dB PSNR improvement on the one
720P clip I tried.

(Also removed "index" which was being updated but not used)

Change-Id: I413bea802b142886bfcf8d8aa7f5a2f0c524fd4b

Previously, Y-adaptive UV intra coding only enabled on key frames in
UVINTRA experiment. This commit enabled the same coding for inter
frames, so the encoding of UV intra modes are consistent cross all
frame types. Tests on derf set showed a very small overall gain around
.04%:

http://www.corp.google.com/~yaowu/no_crawl/interUVintra.html

The gain looks to be reasonable given inta coded MBs is only a
small portion of MBs in inter frames.

Change-Id: Ic6fc261923f2c253f4a0c9f8bccf4797557b9e16

This commit removed the macro CONFIG_I8X8, which was used to indicate
the 8x8 intra prediction experiment, made the change fully merged in.

Change-Id: Iafa4443781ce6e83f5591c12ba615a0e92ce0ea0

vp8_mode_contexts[] is an entropy table used to code inter mode
choices. It was a fixed constant table. This commit made the entropy
context adaptive. Tests on derf set showed very good consistent gains
on all metrics: avg psnr .47%, overall psnr .46% and ssim .40%.

http://www.corp.google.com/~yaowu/no_crawl/newModeContext.html

Change-Id: Ia62b14485c948e2b74586118619c5eb2068b43b2

fixed issues caused by conflicts between two experiments.

Change-Id: I56a9bd69493e4850c121ea057a6233c55777c2a5

This patch introduces the concept of dual inter16x16 prediction. A
16x16 inter-predicted macroblock can use 2 references instead of 1,
where both references use the same mvmode (new, near/est, zero). In the
case of newmv, this means that two MVs are coded instead of one. The
frame can be encoded in 3 ways: all MBs single-prediction, all MBs dual
prediction, or per-MB single/dual prediction selection ("hybrid"), in
which case a single bit is coded per-MB to indicate whether the MB uses
single or dual inter prediction.

In the future, we can (maybe?) get further gains by mixing this with
Adrian's 32x32 work, per-segment dual prediction settings, or adding
support for dual splitmv/8x8mv inter prediction.

Gain (on derf-set, CQ mode) is ~2.8% (SSIM) or ~3.6% (glb PSNR). Most
gain is at medium/high bitrates, but there's minor gains at low bitrates
also. Output was confirmed to match between encoder and decoder.

Note for optimization people: this patch introduces a 2nd version of
16x16/8x8 sixtap/bilin functions, which does an avg instead of a
store. They may want to look and make sure this is implemented to
their satisfaction so we can optimize it best in the future.

Change-ID: I59dc84b07cbb3ccf073ac0f756d03d294cb19281