Integrate intra mode mask speed feature with the mode_skip_mask
scheme. Move it outside the mode search loop in the
vp9_rd_pick_inter_mode_sb function.

Change-Id: I7738fea749bfdc08ad05d7f2524feb8ff67568d9

This speed feature is used in real-time setting only. Remove the
related condition check in the rate-distortion optimization search
loop.

Change-Id: Iaacc1e268214634e6f95c5048c28a60cec6c42fc

Unify this speed feature in the ref_frame_skip_mask scheme.

Change-Id: I7ea5646da02d3ea643680c22d50dabd448d55a27

Change-Id: I0da29c858c6c1eb5ef07cee8f599329f5a002da9

Refactor overlay frame speed-up related function. Make it unified
with the ref_frame_skip_mask system and Move it out of the mode
search loop.

Change-Id: I0dde9baf44354f6ba00b4679cba02fa6a30c7316

This commit refactor the rate-distortion optimization search for
regular block sizes to remove the speed feature dependency on mode
search order.

Change-Id: Ied033ee484c2957e17baa7b6450b720fe7dd0e7d

This commit fixes a bug related to skipping intra mode checking, by
using a separate variable to store the best prediction error from
inter mode. It avoids unintentionally overwriting intra mode
rate-distortion cost, and hence affecting other speed features.

Change-Id: I99e12993339c84c8b4f597996b372012e5858fae

Assigning selected reference frame pointer is done in the
encode_superblock function. No need to do this at the end of
rate-distortion optimization search.

Change-Id: I33fcede0fd304b4a4c4deef2d126d79546a9c070

This commit refactors the vp9_rd_pick_inter_mode_sb function to
remove the intra mode early termination dependency on the mode
search order.

Change-Id: If6ac49aa7c530c7b9a5bd31b0ab84db83e192bec

This commit allows the encoder to find current best prediction mode
state using best_mbmode, instead of fetching from the static mode
search table via best_mode_index.

Change-Id: Ibefeab83aed33a49c2be03e83f09153856ca4271

The rate costs calculated for inter modes are not precise in some
cases, which causes NEWMV is chosen instead of NEARESTMV, NEARMV,
and ZEROMV. This patch added checks for these cases, and corrected
the mode decisions.

Borg tests at speed 3 showed:
1. stdhd set: 0.102% PSNR gain and 0.088% SSIM gain.
2. derf set:  0.147% PSNR gain and 0.132% SSIM gain.
No speed change.

Change-Id: I35d17684b89ad4734fb610942d707899146426db

This commit turns on adaptive motion search for ARF coding, in
addition to other normal inter frame coding. It improves the
average compression efficiency:

stdhd 0.1%
derf  0.04%

For the test sequences, the speed 3 runtime is reduced:

pedestrian 1080p 2000 kbps, 149932 ms -> 144580 ms, (3.3% speed-up)
bus CIF 1000 kbps, 8050 ms -> 7895 ms, (1.9%)
highway CIF 100 bkps, 45033 ms -> 44078 ms, (2.2%)

Change-Id: I5228565b609f99e8ae04f6140a2bf2b64a831d21

This commit allows the encoder to store outcomes of single reference
frame modes and compares them to decide if the inter prediction
filter, forward transform, and quantization can be skipped.

The compression performance of speed 3 is down
derf  -0.364%
stdhd -0.198%

For test sequences, the speed 3 runtime is reduced
highway CIF 100 kbps, 51976 ms -> 45033 ms, 13% speed-up
stockholm 720p 1000 kbps, 71826 ms -> 67838 ms, 5.5% speed-up
pedestrian 1080p 2000 kbps, 154924 ms -> 150702 ms, 2.6% speed-up

Change-Id: I5aa26f918d2b4b5197a2c0afa2779319f1c88e44

intra_super_block_yrd() and inter_super_block_yrd() are largely same,
this commit merges them into one to reduce code duplication.

Change-Id: I64d7042a5b099345627cf55663010c185b25ec37

This commit allows the encoder to skip check on compound inter
modes in the rate-distortion optimization loop, if the reference
frame bias signs are the same.

Change-Id: Ib753e6bb11cbdd338aee69dbe2b649671f75a6b0

This commit skips the compound inter mode prediction check in the
rate-distortion optimization loop for ARF coding. It reduces the
runtime for certain test clips at speed 3, at no compression
performance change:

bus CIF 1000 kbps, 8260 ms -> 8090 ms, 1.8% speed-up
stockholm 720p 1000 kbps, 74453 ms -> 71826 ms, 2.9% speed-up

No visible speed-up for pedestrian area 1080p at 2000 kbps.

Change-Id: Ic68aa56837159b726563b784e2e3729e846465ad

Use unsigned int type to store the sse in the pixel domain. The
precision is sufficient to handle sse of block size up to 64x64.
The transform domain version however needs int64_t, since there is
a transfer gain applied in the forward transformation that might
cause unsigned int overflow.

Change-Id: Ifef97c38597e426262290f35341fbb093cf0a079

This commit allows encoder to skip intra coding mode test, when
the known inter residual is less than the source variance. It
reduces the runtime of speed 3 for test clips:
bus cif 1000 kbps: 8587 ms -> 8260 ms, 3.8% speed-up
pedestrian 1080p 2000 kbps: 161381 ms -> 155241 ms, 3.7% speed-up.

The compression performance is down by
derf   -0.36%
stdhd  -0.25%

Change-Id: I75ce1e035b4da2153cb1ac14111d1a07c05a735d

This commit extends the sse and forward transform computation flag
to support the case 64x64 blocks where there are 4 32x32 2D-DCT
blocks.

Change-Id: I86a3e805dfaa0f3abd812f590520c71aa0e40473

In the partition search, the encoder checks all possible
partitionings in the superblock's partition search tree.
This patch proposed a set of criteria for partition search
early termination, which effectively decided whether or
not to terminate the search in current branch based on the
"skippable" result of the quantized transform coefficients.
The "skippable" information was gathered during the
partition mode search, and no overhead calculations were
introduced.

This patch gives significant encoding speed gains without
sacrificing the quality.

Borg test results:
1. At speed 1,
   stdhd set: psnr: +0.074%, ssim: +0.093%;
   derf set:  psnr: -0.024%, ssim: +0.011%;
2. At speed 2,
   stdhd set: psnr: +0.033%, ssim: +0.100%;
   derf set:  psnr: -0.062%, ssim: +0.003%;
3. At speed 3,
   stdhd set: psnr: +0.060%, ssim: +0.190%;
   derf set:  psnr: -0.064%, ssim: -0.002%;
4. At speed 4,
   stdhd set: psnr: +0.070%, ssim: +0.143%;
   derf set:  psnr: -0.104%, ssim: +0.039%;

The speedup ranges from several percent to 60+%.
                 speed1    speed2    speed3    speed4
(1080p, 100f):
old_town_cross:  48.2%     23.9%     20.8%     16.5%
park_joy:        11.4%     17.8%     29.4%     18.2%
pedestrian_area: 10.7%      4.0%      4.2%      2.4%
(720p, 200f):
mobcal:          68.1%     36.3%     34.4%     17.7%
parkrun:         15.8%     24.2%     37.1%     16.8%
shields:         45.1%     32.8%     30.1%      9.6%
(cif, 300f)
bus:              3.7%     10.4%     14.0%      7.9%
deadline:        13.6%     14.8%     12.6%     10.9%
mobile:           5.3%     11.5%     14.7%     10.7%

Change-Id: I246c38fb952ad762ce5e365711235b605f470a66

Updates the vp9_pattern_search function to return integer one-away
neighbors' sad values, for subsequent use in speeding up the
sub-pel search. Also, removes code for the do_refine option
which is not being used currently.
Updates the integer and subpel functions to pass in a 5-element
sad list for output or input.

A new pruned sub-pel search algorithm is implemented that uses
the sad returned from the integer pel search. But it is not
deployed yet.

Change-Id: Ifa9f5ad024b5b660570366d2bd900343e1891520

This commit re-work the operation flow related to prediction
residual generation and the rate-distortion modeling. It saves one
call for model_rd_for_sb.

Change-Id: Icaf96c0ff09c903637ed5283448afe01d798195f

The value of switchable rate has been stored in a local variable.
This change skips the second call to vp9_get_switchable_rate() by
reusing the local variable.

Change-Id: Ib7d3fef7621cc4bde94c6d6e6b3a71f1fd4559f2

Check the mode and motion vector cost. If it is already above
the existing best rate-distortion cost, skip the rest check process
on this mode.

Change-Id: Ie065cebdfda2a3be3be18b8e8b43dc29aaa8c179

This commit makes the rate distortion modeling run in the unit of
maximum transform block size. No compression/speed change observed.
It is for the use of later fast forward transform purpose.

Change-Id: Ibaaedb69c765e8d0c5d5012f0ec07f36fd9f68fd

This commit addes a new strategy to reduce the search for optimal
interpolation filter type. The encoder counts and store how many each
filter type is selected and used for each of the reference frames.
A filter type that is rarely used for all three reference frames is
masked out to avoid computation.

The impact on compression is neglectible:
-0.02% on derf
+0.02% on stdhd

Encoding time is seen to reduce by 2~3%.

Change-Id: Ibafa92291b51185de40da513716222db4b230383

The variable is never read.

Change-Id: I94141c1667fa5d10604cd6f83c5f64df107dee94

Change-Id: Iccd7cec885e8aeb0e54613d888f9960c393cee0b

Change-Id: Id68736ffa71843b02ee3634edc525219d6e54d4f

In the full-rd transform size search, we go through all transform
sizes to choose the one with best rd score. In this patch, an
early termination is added to stop the search once we see that the
smaller size won't give better rd score than the larger size. Also,
the search starts from largest transform size, then goes down to
smallest size.

A speed feature tx_size_search_breakout is added, which is turned off
at speed 0, and on for other speeds. The transform size search is
turned on at speed 1.

Borg test results:
1. At speed 1,
   derf set: psnr gain: 0.618%, ssim gain: 0.377%;
   stdhd set: psnr gain: 0.594%, ssim gain: 0.162%;
   No noticeable speed change.
3. At speed 2,
   derf set: psnr loss: 0.157%, ssim loss: 0.175%;
   stdhd set: psnr loss: 0.090%, ssim loss: 0.101%;
   speed gain: ~4%.

Change-Id: I22535cd2017b5e54f2a62bb6a38231aea4268b3f

This commit enables the encoder to record the location of the
center frame to generate alter reference frame. It then allows to
skip checking prediction modes of other reference frame types when
it comes to encode this frame.

The speed 3 runtime is reduced for the test sequences:
bus at CIF 1000 kbps, 9791 ms -> 9446 ms, i.e., 3.5% speed-up,
pedestrian at 1080p 2000 kbps, 184043 ms -> 175730 ms, i.e., 4.5%
speed-up.

No compression performance change observed.

Change-Id: Iacfde3bcc1445964e7a241f239bd6ea11cb94bd1

Change-Id: Ib52e1ce5aa4a22d70e124c3c29a92e00ffac50e4

Change-Id: I4f0bac3367b1a7c5e7f746b97ca619b81c8dfc49

This commit enables the encoder to skip NEARMV and ZEROMV if the
above and left blocks have identical reference frame, and the
current reference is different from that. It reduces the runtime
of speed 3 for test sequences:
bus cif at 1000 kbps 10064 ms -> 9823 ms
pedestrian 1080p at 2000 kbps 193078 ms -> 189559 ms

The compression performance is changed by
derf  -0.085%
stdhd -0.103%

Change-Id: If304f26d42e6412152a84c3dd7b02635c38444f4

This commit allows the encoder to check the above and left neighbor
blocks' reference frames and motion vectors. If they are all
consistent, skip checking the NEARMV and ZEROMV modes. This is
enabled in speed 3. The coding performance is improved:

pedestrian area 1080p at 2000 kbps,
from  74773 b/f, 41.101 dB, 198064 ms
to    74795 b/f, 41.099 dB, 193078 ms

park joy 1080p at 15000 kbps,
from 290727 b/f, 30.640 dB, 609113 ms
to   290558 b/f, 30.630 dB, 592815 ms

Overall compression performance of speed 3 is changed
derf  -0.171%
stdhd -0.168%

Change-Id: I8d47dd543a5f90d7a1c583f74035b926b6704b95

This commit enables encoder to select fast forward transform and
quantization path according to the prediction residual sse/variance,
in the rate-distortion optimization scheme.

Change-Id: Ief9fc3844fd4107166d401970e800c6e5ce2b5fe

Change-Id: I303213fbb0ad5fbf28f0ca17271624b75e9d6319

Change-Id: Ie813af24d23c7d72b38749b3382b315daf516be5

Change-Id: I1b3c5337f018dde27dc819ab18bd081d169a91e8

Moved encode_breakout code out of handle_inter_mode().

Change-Id: Icd661136b05fdf163768c406f91e0c98a8df89eb