In the variance calculations the difference is summed and later squared.
When the sum exceeds sqrt(2^31) the value is treated as a negative when
it is shifted which gives incorrect results.

To fix this we force the multiplication to be unsigned.

The alternative fix is to shift sum down by 4 before multiplying.
However that will reduce precision.

For 16x16 blocks the maximum sum is 65280 and sqrt(2^31) is 46340 (and
change).

This change is based on:
16982342 Missed some variance casts
fea3556e Fix variance overflow

Change-Id: I2c61856cca9db54b9b81de83b4505ea81a050a0f

Change-Id: Iedaa108ddb65f54d768424f9c47ad4d069b656fd

Change-Id: Ic084c475844b24092a433ab88138cf58af3abbe4

For non-static functions, change the prefix to vp9_. For static functions,
remove the prefix. Also fix some comments, remove unused code or unused
function prototypes.

Change-Id: I1f8be05362f66060fe421c3d4c9a906fdf835de5

This change encompasses VP8_PTR, VP8_COMP, VP8D_COMP, VP8_COMMON,
VP8Decompressor and VP8Common.

Change-Id: I514ef4ad4e682370f36d656af1c09ee20da216ad

For local symbols, make them static instead.

Change-Id: I13d60947a46f711bc8991e16100cea2a13e3a22e

Change-Id: Ic5a5f60e1ff9d9ccae4174160d36529466eeb509

As suggested by Paul, this commit separate the subpel refmv selection
into a separate experiment. It also changed a couple variable names
to better reflect the nature of the variables.

Change-Id: Id951c3cadc61a982dd15afe641000f60213b8995

Previously, in evaluating reference motion vectors, MVs are always
rounded to integer pixel position and SADs are calculated.  This
commit takes into account the subpixel portion of the mvs, and uses
bilinear interpolation to produce reference pixel values in subpixel
postions. In addition, SSE is used in place of SAD. Pixels used are
16x2 above and 2x16 to the left.

This commmit intends to test the potential of this line of work in
term of compression improvement, obviously, the change would increase
decoder complexity significantly.

Test results
std-hd: 1.738%(avg) 1.779%(glb), 1.663%(ssim)
derf: 0.472%(avg) 0.477%(glb), 0.418%(ssim)

Change-Id: I3ae1b098f6289df78891134d9a5e4bb2fde87a0b

Coding and costing of mv reference signal.

Issues in updating MV ref with COMPANDED_MVREF_THRESH
to be resolved. Ideally the MV precision should be defined based
on absolute MV magnitude not as now the MV ref magnitude.

Update to mv counts moved into bitstream.c because otherwise
if the motion reference is changed at the last minute the encoder
and decoder get out of step in terms of the counts used to update
entropy probs.

Code working on a few test clips but no results yet re benefit vs
signaling cost and no tuning of red loop to test lower cost alternatives
based on the available reference values.

Patch 3. Added check to make sure we don't pick a reference
that would give rise to an uncodeable / out of range residual.

Patch 6-7: Attempt to rebase. OK to submit but best to leave flag off for now.

Patch 9. Remove print no longer needed.

Change-Id: I1938c2ffe41afe6d3cf6ccc0cb2c5d404809a712

Removes the old mv encoding code.

Change-Id: I1026d48cc5ac15ccb776f98463e929254c3dc7da

The previous double if and else code snippets were rather confusing.

Change-Id: Id1b6152fa0e471beb9b20407aa406e109c1471e3

Change-Id: I5bca7b7a4b230082d36ac6fb84db84137ad177d7

The commit changed to avoid using pixels from extended border in
in evaluating and select best reference motion vector.

Change-Id: I39b758889373e42ed2889d59744388e5b9c1a20a

The commit changed to use 3 rows above and 3 cols from left for SAD
scoring for selecting the best reference motion vector. The change
helped std-hd set by >.2% on psnr/ssim metrics.

Change-Id: Ifad3b528d0b4b6e3c22518af789d76eff23c1520

The high-precision (1/8) pel bit is turned off if the reference
MV is larger than a threshold. The motivation for this patch is
the intuition that if motion is likely large (as indicated by
the reference), there is likley to be more motion blur, and as
a result 1/8 pel precision would be wasteful both in rd sense
as well as computationally.

The feature is incorporated as part of the newmventropy experiment.
There is a modest RD improvement with the patch. Overall the
results with the newmventropy experiment with the threshold being
16 integer pels are:

derf: +0.279%
std-hd: +0.617%
hd: +1.299%
yt: +0.822%

With threshold 8 integer pels are:

derf: +0.295%
std-hd: +0.623%
hd: +1.365%
yt: +0.847%

Patch: rebased
Patch: rebase fixes

Change-Id: I4ed14600df3c457944e6541ed407cb6e91fe428b

This commit merges those parts of the CONFIG_NEW_MVREF
that specifically relate to choosing a better set of candidate
MV references into the NEWBESTREFMV experiment.

CONFIG_NEW_MVREF will then be used for changes relating
to the explicit coding of a cost optimized MV reference in the
bitstream as part of MV coding.

Change-Id: Ied982c0ad72093eab29e38b8cd74d5c3d7458b10

Extend experiment to use both vectors from MBs
coded using compound prediction as candidates.

In final sort only consider best 4 candidates
for now but make sure 0,0 is always one of them.

Other minor changes to new MV reference code.
Pass in Mv list to vp8_find_best_ref_mvs().

Change-Id: Ib96220c33c6b80bd1d5e0fbe8b68121be7997095

Alternative strategy for finding a list of candidate motion
vectors to use as reference values in mv coding and as
nearest and near.

Sort by sad in vp8_find_best_ref_mvs() rather than just
pick the best. Allow 0,0 as a best ref option but not a
nearest or near unless there are no alternatives.

Encode/Decode verified on at least some clips.

Some commented out experimental and stats code still in place.

Gain over existing code averages about 1% on derf (alll metrics)
with improvement on all clips. Other test results pending.

The entropy coding of the mode (nearest/near etc) still
depends upon and requires the old "findnear" code so
this needs looking at and may provide room for further gains.

Change-Id: I871d7cba1d1c379c4bad9bcccce1fb19c46b8247

Change-Id: I01dd96da18fe22450bb372b10ba90eab52e29bdf

The reference motion vector selected by surrounding pixels that has
the best matching score is used as nearest motion vector.

The change has shown consistent gain on all test sets, compression
gains range from .2% to .6%. The variation is largely dependent on
various other experiments on or off.

Change-Id: I5552e1c2f6fc57c3e8818a5ee41ffda89af05e75

Change-Id: I7f531d55f076254c84402fed34af93fe3685fbdf

Change-Id: Id38baf1b89648ef534e28be72f583137871f920c

Using surrounding reconstructed pixels from left and above to select
best matching mv to use as reference motion vector for mv encoding.

Test results:
       AVGPSNR  GLBPSNR VPXSSIM
Derf:  1.107%   1.062%  0.992%
Std-hd:1.209%   1.176%  1.029%

Change-Id: I8f10e09ee6538c05df2fb9f069abcaf1edb3fca6

Merged in the high_precision_mv experiment to make it easier
to work on new mv encoding strategies. Also removed
coef_update_probs3().

Change-Id: I82d3b0bb642419fe05dba82528bc9ba010e90924

Approximate the Google style guide[1] so that that there's a written
document to follow and tools to check compliance[2].

[1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml
[2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py

Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f

Incorporates mv_ref, mbsplit and second_mv into the adaptive
entropy framework. The mv_ref framework has been modified from
before.

Adds some clean-ups and fixes.

Results with the adaptive entropy experiment are currently up by
+1.93% on derf; +2.33% std-hd and +1.87% yt-hd.

Fixed a nasty intermittent bug.

Change-Id: I4b1ac9f9483b48432597595195bfec05f31d1e39

This is the first patch to add superblock (32x32) coding
order capabilities. It does not yet do any mode selection
at the SB level, that will follow in a further patch.

This patch encodes rows of SBs rather than
MBs, each SB contains 2x2 MBs.

Two intra prediction modes have been disabled since they
require reconstructed data for the above-right MB which
may not have been encoded yet (e.g. for the bottom right
MB in each SB).

Results on the one test clip I have tried (720p GIPS clip)
suggest that it is somewhere around 0.2dB worse than the
baseline version, so there may be bugs.

It has been tested with no experiments enabled and with
the following 3 experiments enabled:
  --enable-enhanced_interp
  --enable-high_precision_mv
  --enable-sixteenth_subpel_uv
in each case the decode buffer matches the recon buffer
(using "cmp" to compare the dumped/decoded frames).
Note: Testing these experiments individually created
errors.

Some problems were found with other experiments but it
is unclear what state these experiments are in:
  --enable-comp_intra_pred
  --enable-newentropy
  --enable-uvintra

This code has not been extensively tested yet, so there
is every likelihood that further bugs remain. I also
intend to do some code cleanup & refactoring in tandem
with the next patch that adds the 32x32 modes.

Change-Id: I1eba7f740a70b3510df58db53464535ef881b4d9

This is a code snapshot of experimental work currently ongoing for a
next-generation codec.

The codebase has been cut down considerably from the libvpx baseline.
For example, we are currently only supporting VBR 2-pass rate control
and have removed most of the code relating to coding speed, threading,
error resilience, partitions and various other features.  This is in
part to make the codebase easier to work on and experiment with, but
also because we want to have an open discussion about how the bitstream
will be structured and partitioned and not have that conversation
constrained by past work.

Our basic working pattern has been to initially encapsulate experiments
using configure options linked to #IF CONFIG_XXX statements in the
code. Once experiments have matured and we are reasonably happy that
they give benefit and can be merged without breaking other experiments,
we remove the conditional compile statements and merge them in.

Current changes include:
* Temporal coding experiment for segments (though still only 4 max, it
  will likely be increased).
* Segment feature experiment - to allow various bits of information to
  be coded at the segment level. Features tested so far include mode
  and reference frame information, limiting end of block offset and
  transform size, alongside Q and loop filter parameters, but this set
  is very fluid.
* Support for 8x8 transform - 8x8 dct with 2nd order 2x2 haar is used
  in MBs using 16x16 prediction modes within inter frames.
* Compound prediction (combination of signals from existing predictors
  to create a new predictor).
* 8 tap interpolation filters and 1/8th pel motion vectors.
* Loop filter modifications.
* Various entropy modifications and changes to how entropy contexts and
  updates are handled.
* Extended quantizer range matched to transform precision improvements.

There are also ongoing further experiments that we hope to merge in the
near future: For example, coding of motion and other aspects of the
prediction signal to better support larger image formats, use of larger
block sizes (e.g. 32x32 and up) and lossless non-transform based coding
options (especially for key frames). It is our hope that we will be
able to make regular updates and we will warmly welcome community
contributions.

Please be warned that, at this stage, the codebase is currently slower
than VP8 stable branch as most new code has not been optimized, and
even the 'C' has been deliberately written to be simple and obvious,
not fast.

The following graphs have the initial test results, numbers in the
tables measure the compression improvement in terms of percentage. The
build has  the following optional experiments configured:
--enable-experimental --enable-enhanced_interp --enable-uvintra
--enable-high_precision_mv --enable-sixteenth_subpel_uv

CIF Size clips:
http://getwebm.org/tmp/cif/
HD size clips:
http://getwebm.org/tmp/hd/
(stable_20120309 represents encoding results of WebM master branch
build as of commit#7a159071)

They were encoded using the following encode parameters:
--good --cpu-used=0 -t 0 --lag-in-frames=25 --min-q=0 --max-q=63
--end-usage=0 --auto-alt-ref=1 -p 2 --pass=2 --kf-max-dist=9999
--kf-min-dist=0 --drop-frame=0 --static-thresh=0 --bias-pct=50
--minsection-pct=0 --maxsection-pct=800 --sharpness=0
--arnr-maxframes=7 --arnr-strength=3(for HD,6 for CIF)
--arnr-type=3

Change-Id: I5c62ed09cfff5815a2bb34e7820d6a810c23183c

This is the first patch for refactoring of the code related to
high-precision mv, so that 1/4 and 1/8 pel motion vectors can
co-exist in the same bit-stream by use of a frame level flag.
The current patch works fine for only use of 1/4th and
only use of 1/8th pel mv, but there are some issues with the
mode switching in between. Subsequent patches on this change Id
will fix the remaining issues.

Patch 2: Adds fixes to make sure that multiple mv precisions can
co-exist in the bit-stream. Frame level switching has been tested
to work correctly.

Patch 3: Fixes lines exceeding 80 char

Patch 4:
http://www.corp.google.com/~debargha/vp8_results/enhinterp.html

Results on derf after ssse3 bugfix, compared to everything
enabled but the 8-tap, 1/8-subpel and 1/16-subpel uv. Overall the
gains are about 3% now. Hopefully there are no more bugs lingering.
Apparently the sse3 bug affected the quartel subpel results more than
the eighth pel ones (which is understandabale because one bad predictor
due to the bug, matters less if there are a lot more subpel options
available as in the 1/8 subpel case).
The results in the 4th column correspond to the current settings.
The first two columns correspond to two settings of adaptive switching
of the 1/4 or 1/8 subpel mode based on initial Q estimate. These
do not work as good as just using 1/8 all the time yet.

Change-Id: I3ef392ad338329f4d68a85257a49f2b14f3af472

This commit merges the NEWNEAR experiment such that it
is effectively always on.

The fact that there were changes in the threading code again
highlights the need to strip out such features during the
bitstream development phase as trying to maintain this code
(especially as it is not being tested) slows the development cycle.

Change-Id: I8b34950a1333231ced9928aa11cd6d6459984b65

vp8_mode_contexts[] is an entropy table used to code inter mode
choices. It was a fixed constant table. This commit made the entropy
context adaptive. Tests on derf set showed very good consistent gains
on all metrics: avg psnr .47%, overall psnr .46% and ssim .40%.

http://www.corp.google.com/~yaowu/no_crawl/newModeContext.html

Change-Id: Ia62b14485c948e2b74586118619c5eb2068b43b2

CNT_INTRA has been used for counting (0,0) motion vectos, this commit
renames it to CNT_ZEROMV

Change-Id: I8f67c5468370090525faf84ba5b3f780d302443f

This is an experiment to include a mv contribution from last frame to
nearest and near mv definition. Initial test showed some small though
consistent gain.

latest patch slightly better result ~.13%-~.18%.

TODO: the entropy used to encode the mode choice, i.e. the mv counts
based conditional distribution of modes should be re-collected to
reflect this change, it is expected that there is some further gain
from that.

Change-Id: Ief1e284a36d8aa56b49ae5b360c91419ec494fa4

Patch 1 to Patch 3 is an initial implementation of 8x8 intra prediction
modes, here are with the following assumptions:
a. 8x8 has 4 prediction modes DC, H, V and TM
b. UV 4x4 block use the same mode as corresponding 8x8 area
c. i8x8 modes are enabled for key frame only for now
Patch 4:
d. removed debug code from previous patches
Patch 5:
e. added stats code to collect entropy stats and further cleaned up
Patch 6:
f. changed mode stats code to collect finer stats of modes
Patch 7:
g. normalized i8x8 modes distribution to total at 256 (8bits).
Patch 8:
h. fixed a bug in decoder and removed debug printf output.
Patch 9:
i. more cleanups to address paul's comment
Patch 10:
j. messy rebase/merges to bring the commit up to date.

Tests on HD clips encoded with all key frame showing consistent gain
on all clips and all metrics:~0.5%(psnr) and 0.6%(ssim):
http://www.corp.google.com/~yaowu/no_crawl/i8x8hd_allkey_fixedq.html

To build and test, configure with:
--enable-experimental --enable-i8x8

Change-Id: I9813fe07ae48cab5fdb5d904bca022514ad01e7f

Declared the bmi in MODE_INFO as a union instead of B_MODE_INFO.
This reduced the memory footprint by 518,400 bytes for 1080
resolutions.  The decoder performance improved by ~4% for the
clip used and the encoder showed very small improvements. (0.5%)
This reduction was first mentioned to me by John K. and in a
later discussion by Yaowu.
This is WIP.

Change-Id: I8e175fdbc46d28c35277302a04bee4540efc8d29

The compiler produces better assembly when using int_mv
for assignments.  The compiler shifts and ors the two 16bit
values when assigning MV.

Change-Id: I52ce4bc2bfbfaf3f1151204b2f21e1e0654f960f

A large number of functions were defined with external linkage, even
though they were only used from within one file. This patch changes
their linkage to static and removes the vp8_ prefix from their names,
which should make it more obvious to the reader that the function is
contained within the current translation unit. Functions that were
not referenced were removed.

These symbols were identified by:

  $ nm -A libvpx.a | sort -k3 | uniq -c -f2 | grep ' [A-Z] ' \
    | sort | grep '^ *1 '

Change-Id: I59609f58ab65312012c047036ae1e0634f795779

Applied same method used in vp8_rd_pick_inter_mode() to improve
the accuracy of MV prediction.

Change-Id: Ia50ae26208b18482695601f32febd99fe89fbc17

Reduce the code size.

Change-Id: I2e1998557a3c8776e262c442fd758c25e17aff7a