Change-Id: I95fab6f51967008acf1bc9e98fdb7bb56974807f

this commit is to add an variable in the macroblock level mode
info structure to track the transform size used in each MB, so
the information can be used later in the loop filter to change
how loop filter works on MBs with different transform sizes.

Change-Id: Id0eeaba6cc854c6d1be00ed8d237b3d9e250e447

Slight tweaks to the new minq equations to bring results more into line with
original lookup tables.

Change-Id: I969fc87d95912df549b6775e83ee2345e84d4da0

Addressed a couple of other QIndex dependencies.

Change-Id: I15b224bffd0210d3c7065cb6905156f2ca8e9ea9

Fixed bug in firspass.c call to vp8_initialize_rd_consts()

This was passing in vp8_dc_quant(cm->base_qindex, cm->y1dc_delta_q)
 instead of (cm->base_qindex + cm->y1dc_delta_q).

It just so happens that for the value 26 used for cm->base_qindex in the
unextended Q case,  the two give similar results. However, when using
the extended Q range the two are very different.

Also added more stats output and partly disabled another broken feature.

Change-Id: Iddf6cf5ea8467c44b7c133f38e629f6ba6f2581e

This is an experiment to include a mv contribution from last frame to
nearest and near mv definition. Initial test showed some small though
consistent gain.

latest patch slightly better result ~.13%-~.18%.

TODO: the entropy used to encode the mode choice, i.e. the mv counts
based conditional distribution of modes should be re-collected to
reflect this change, it is expected that there is some further gain
from that.

Change-Id: Ief1e284a36d8aa56b49ae5b360c91419ec494fa4

This comitt brings accross changes from the public branch
commit number Icf74d13af77437c08602571dc7a97e747cce5066.

The main puurpose of this comit relates to CQ mode but it
also includes some refactoring of the two pass code which
I hope will make tuning the experimental branch for the new
quantizer range a little less painfull.

Change-Id: I278e989436a928fc1fe7761068960048f9d7a376

This commit resolves further QIndex look up tables to facilitate
experimentation with the quantizer range.

In some cases  rather than remove the look up tables completely
I have created functions that are called once  to populate them
using a formulaic approach base on the actual quantizer.

The use of these functions based on best fit of data from the original
tables does affect the results on some clips but across the derf test
set the effect was broadly neutral.

Change-Id: I8baa61c97ce87dc09a6340d56fdeb681b9345793

One of the problems arising when tweaking or adjusting the quantizer
tables is that there are a lot of look up tables that depend on the QINDEX.
Any adjustment to the link between QINDEX and real quantizer therefore tends
to break aspects of for example the rate control.

In this check in I have replaced several of the look up tables with functions that
approximate the same results as the old Q luts but use a formulaic approach
based on real Q values rather than QIndex. This should hopefully make it easier
to experiment with changes to the Q tables without always having to go through
and hand optimize a set of look up tables. Once things stabilize we may choose
to re-instate luts for the sake of performance.

Patch 2:
    Addressed Ronald's comments.
    vp8_init_me_luts() Added so luts only initialized once.

Change-Id: Ic80db2212d2fd01e08e8cb5c7dca1fda1102be57

Corrected dc lookup table to maintain ac/dc balance
close to what it was previously.

Firstpass not being passed the adjusted Q index for
the extended range.

Change-Id: Ic0200dabda445fea03bf81067999cb2670e99b77

The buffer level was able to increase indefinitely rather than
being clipped to the maximum buffer size specified by the user.

This change checks the buffrer level and prevents it from
going beyond the upper limit of the buffer.

Change-Id: Ifff55f79d3c018e4d3d77e554b11ada543cc1654

Removal of CONFIGURE_SEGMENTATION ifdefs.

Removal of legacy support code fo the old coding mechanism.

Use local reference "xd" for MACROBLOCKD structure in
encode_frame_to_data_rate()

Moved call to choose_segmap_coding_method() out of encode
loop as the cost of segmentation is not properly accounted
in the loop anyway. If this is desirable in the future it
can be moved back. The use of this function to do all the
analysis and set the probabilities also removes the need
to track segment useage in threading code.

Change-Id: I85bc8fd63440e7176c73d26cb742698f9b70cade

This check in includes quite a lot of clean up and refactoring.

Most of the analysis and set up for the different coding options for the
segment map (currently simple distribution based coding or temporaly
predicted coding), has been moved to one location (the function
choose_segmap_coding_method() in segmenation.c). This code was previously
scattered around in various locations making integration with other
experiments and modification / debug more difficult.

Currently the functionality is as it was with the exception that the
prediction probabilities are now only transmitted when the temporal
prediction mode is selected.

There is still quite a bit more clean up work that will be possible
when the #ifdef is removed. Also at that time I may rename and alter
the sense of macroblock based variable "segment_flag" which indicates
(1 that the segmnet id is not predicted vs 0 that it is predicted).

I also intend to experiment with a spatial prediction mode that can be
used when coding a key frame segment map or in cases where temporal
prediction does not work well but there is spatial correlation.

In a later check in when the ifdefs have gone I may also move the call
to choose_segmap_coding_method() to just before where the bitsream is
packed (currently it is in vp8_encode_frame()) to further reduce the
possibility of clashes with other experiments and prevent it being called
on each itteration of the recode loop.

Change-Id: I3d4aba2a2826ec21f367678d5b07c1d1c36db168

Added last_segmentation_map[] structure
to keep track of what we had before when
doing temporal prediction. With this change
the existing code does once again appear to
be giving a decodable bitstream for both
temporal and standard prediction modes.
However, it is still somewhat messy and
confused and there is no option to take
advantage of spatial prediction so it could
do with further work.

Some housekeeping / clean out.

Change-Id: I368258243f82127b81d8dffa7ada615208513b47

For ease of testing and merging experiments I have
removed in line code in encode_frame() that assigns
MBs to be t8x8 or t4x4 coded segments and have
moved the decision point and segment setup to
the init_seg_features0 test function.

Keeping everything in one place helps make sure
for now that experiments using segmentation are
not fighting each other.

Also made sure mode selection code can't choose 4x4
modes if t8x8 is selected.

Patch2: In init_seg_features() add checks
for SEG_LVL_TRANSFORM active.

Change-Id: Ia1767edd99b78510011d4251539f9bc325842e3a

Removed code in #if CONFIG_SEGMENTATION that
enables segmentation and creates a test segmentation
map, to avoid conflicts with the other segmentation test
code,

Change-Id: I7a21a44ed188b814cd80b30dd628c62474eba730

Initial attempt at using new segment feature signaling
to indicate 4x4 or 8x8 transform.

needs --enable-experimental --enable-t8x8

Note this is work in progress.

Change-Id: Ib160d46a5d810307bfcbc79853ce1a65b5b870b7

Temporary check in to turn off other segment features
tests when #if CONFIG_T8X8 is set as the assignment of
MBs to differnt segments in each case  will conflict.

The 8x8 code will be modified to use the new segment
feature method properly in a later check in.

Increase bits allowed for EOB end stop marker to 6 ready
for 8x8.

Change-Id: I4835bc8d3bf98e1775c3d247d778639c90b01f7f

No change to functionality or output.

Updates to the segment feature data structure now all done
through functions such as set_segdata() and get_segdata()
in seg_common.c.

The reason for this is to make changing the structures (if needed)
and debug easier.

In addition it provides a single location for subsequent addition
of range and validity checks. For  example valid combination of
mode and reference frame.

Change-Id: I2e866505562db4e4cb6f17a472b25b4465f01add

This commit tries to do UV intra mode coding adaptive to Y intra mode.
Entropy context is defined as conditional PDF of uv intra mode given
the Y mode. All constants are normalized with 256 to be fit in 8 bits.

This provides further coding efficiency beyond the quantizer adaptive
y intra mode coding. Consistent gains were observed on all clips and
all bit rates for HD all key encoding tests.

To test, configure with
--enable-experimental --enable-uvintra

Change-Id: I2d78d73f143127f063e19bd0bac3b68c418d756a

Removal of configure #ifdefs so that segment features
always available. Removal of code supporting old
segment feature method.

Still a good deal of tidying up to do.

Change-Id: I397855f086f8c09ab1fae0a5f65d9e06d2e3e39f

Modify reference frame segmentation so that ONE or MORE
reference frames may be marked as a available for a given
segment.

Fixed bugs relating to segment coding of INTRA and some
INTER modes at the segment level.

Modified Q boost for static areas based on ambient average Q.

Strong results now on clips with significant static areas.
(some data points in derf set as high as 9% and some static &
slide show type content in YT set > 20%)

Change-Id: Ia79f912efa84b977f35a23683ae3643251e24f0c

Some correction for entropy impact of segment signaled (EOB and ref frame)

Other slight tweaks.

Derf VBR average gain now over 1% (best over 7%)
One YT test clip has gains of circa 30% (VBR)

There is still an issue with noisy clips where making the background static
and coded with 0,0 can have a negative effect, especially at low Q.
This is probably because of the loss of smoothing by fractional pixel filters.

Change-Id: I7a225613c98067b96f8fc7a7e36f95d465b2b834

Only encode sign bit for feature data that can have a sign.

Tweaks to the test segmentation rules so that it now actually gives
a net benefit on the derf set of about 0.4% though much higher
on some clips at the low end.

Change-Id: I8e61f1aebf41c9037db7e67e2f8975aa18a0c986

This quite large check in includes the following:

Merge in some code from Ronald (mbgraph.c) that scans a Gf/arf group.
This is used as a basis for a simple segmentation for the normal frames
in a gf/arf group. This code also uses satd functions from Yaowu.

Adds functionality for coding the latest possible position of an EOB for
blocks in the segment. (Currently 0-15 only, hence just for 4x4 dct).
Where the EOB position is 0 this acts like "skip" and the normal coding
of skip at the per mb level is disabled.

Added functions (seg_common.c) for setting and reading segment feature
elements. These may want to be optimized away at some point but while the
mecahnism is in a state of flux they provide a single location for making
changes and keep things a bit cleaner.

This is still proof of concept code. Currently the tested feature set:-

Quantizer,
Loop Filter level,
Reference frame,
Prediction Mode,
EOB end stop.

TBD:-

Add functions for setting and reading the feature data with range
and validity checking.

Handling of signed and unsigned feature data. At the moment all is assumed
to be signed and a sign bit is coded but many cannot be negative.

Correct handling of EOB feature with intra coded blocks.

Testing/trapping of legal/illegal ref frame and mode combinations.

Transform size switch plus merge and test with 8c8 DCT work

Merge and test with Sumans Segmenation coding optimizations

Change-Id: Iee12e83661c7abbd1e0ce6810915eb4ec35e2d8e

Proof of concept test code that encodes mode and reference
frame data at the segment level.

Decode-able bit stream but some issues not yet resolved.
As it this helps a little on a couple of clips but hurts on most as
the basis for segmentation is unsound.

To build and test, configure with
--enable-experimental --enable-segfeatures

Change-Id: I22a60774f69273523fb152db8c31f4b10b07c7f4

Patch 1 to Patch 3 is an initial implementation of 8x8 intra prediction
modes, here are with the following assumptions:
a. 8x8 has 4 prediction modes DC, H, V and TM
b. UV 4x4 block use the same mode as corresponding 8x8 area
c. i8x8 modes are enabled for key frame only for now
Patch 4:
d. removed debug code from previous patches
Patch 5:
e. added stats code to collect entropy stats and further cleaned up
Patch 6:
f. changed mode stats code to collect finer stats of modes
Patch 7:
g. normalized i8x8 modes distribution to total at 256 (8bits).
Patch 8:
h. fixed a bug in decoder and removed debug printf output.
Patch 9:
i. more cleanups to address paul's comment
Patch 10:
j. messy rebase/merges to bring the commit up to date.

Tests on HD clips encoded with all key frame showing consistent gain
on all clips and all metrics:~0.5%(psnr) and 0.6%(ssim):
http://www.corp.google.com/~yaowu/no_crawl/i8x8hd_allkey_fixedq.html

To build and test, configure with:
--enable-experimental --enable-i8x8

Change-Id: I9813fe07ae48cab5fdb5d904bca022514ad01e7f

Plumbing for tuning new segment features on and off.

Change-Id: If86cd6f103296b73030e8af7cf85c5b9bbffdbaf

Some basic plumbing added for a range of segment level features.
MB_LVL_* changed to SEG_LVL_* to better reflect meaning.

Change-Id: Iac96da36990aa0e40afc0d86e990df337fd0c50b

This change makes sure that no key frame recoding in real-time mode
even if CONFIG_REALTIME_ONLY is not configured.

Change-Id: Ifc34141f3217a6bb63cc087d78b111fadb35eec2

When active map is specified and the current frame is not a key frame,
golden frame nor a altref frame then copy only those active regions.

This significantly reduces encoding time by as much as 19% on the test
system where realtime encoding is used. This is particularly useful
when the frame size is large (e.g. 2560x1600) and there's only a few
action macroblocks.

Change-Id: If394a813ec2df5a0201745d1348dbde4278f7ad4

This reverts commit b5ea2fbc. Further
testing showed noticable keyframe popping in some cases, reverting this
for now to give time for a proper fix.

Conflicts:

	vp8/encoder/onyx_if.c
	vp8/encoder/ratectrl.c

Change-Id: I159f53d1bf0e24c035754ab3ded8ccfd58fd04af

This patch fixes a bug in the interaction between the recode loop and
spatial resampling. If the codec was in a spatial resampling state,
and a subsequent iteration of the recode loop disables resampling,
then the source buffer must be reset to the unscaled source.

Change-Id: I4e4cd47b943f6cd26a47449dc7f4255b38e27c77

Change-Id: I7e6bc28e7974a376da747300744e0dd5dc1d21e9

Removes mixed usage of (unsigned) long long and INT64.
Fixes Issue #208.

Change-Id: I220d3ed5ce4bb1280cd38bb3715f208ce23cf83a

sharpness was not recalculated in vp8cx_pick_filter_level_fast

remove last_filter_type. all values are calculated, don't need to update
the lfi data when it changes.

always use cm->sharpness_level. the extra indirection was annoying.

don't track last frame_type or sharpness_level manually. frame type
only matters for motion search and sharpness_level is taken care of in
frame_init

move function declarations to their proper header

Change-Id: I7ef037bd4bf8cf5e37d2d36bd03b5e22a2ad91db

In sub-pixel motion search, the search range is small(+/- 3 pixels).
Preload whole search area from reference buffer into a 32-byte
aligned buffer. Then in search, load reference data from this buffer
instead. This keeps data in cache, and reduces the crossing cache-
line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux)
showed encoder speed improvement:
  3.4%   at --rt --cpu-used =-4
  2.8%   at --rt --cpu-used =-3
  2.3%   at --rt --cpu-used =-2
  2.2%   at --rt --cpu-used =-1

Test on Atom notebook showed only 1.1% speed improvement(speed=-4).
Test on Xeon machine also showed less improvement, since unaligned
data access latency is greatly reduced in newer cores.

Next, I will apply similar idea to other 2 sub-pixel search functions
for encoding speed > 4.

Make this change exclusively for x86 platforms.

Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f

With this fix, the experimental branch now builds and encodes correctly
with the following two configure options respectively:
--enable-experimental --enable-t8x8
--enable-experimental

Change-Id: I3147c33c503fe713a85fd371e4f1a974805778bf

This is done by expanding luma row to 32-byte alignment, since
 there is currently a bunch of code that assumes that
 uv_stride == y_stride/2 (see, for example, vp8/common/postproc.c,
 common/reconinter.c, common/arm/neon/recon16x16mb_neon.asm,
 encoder/temporal_filter.c, and possibly others; I haven't done a
 full audit).
It also uses replaces the hardcoded border of 16 in a number of
 encoder buffers with VP8BORDERINPIXELS (currently 32), as the
 chroma rows start at an offset of border/2.
Together, these two changes have the nice advantage that simply
 dumping the frame memory as a contiguous blob produces a valid,
 if padded, image.

Change-Id: Iaf5ea722ae5c82d5daa50f6e2dade9de753f1003

Please refer to previous commit messages for detailed info:
https://on2-git.corp.google.com/g/#change,5940
https://on2-git.corp.google.com/g/#change,6045

Change-Id: I8b16992f2f69c5a808ad40a3e32ef589cce7c59d