GitLab

* changes:
  vp9: correct context buffer resize check
  vp9: fail decode if block/frame refs are corrupt

The rate costs calculated for inter modes are not precise in some
cases, which causes NEWMV is chosen instead of NEARESTMV, NEARMV,
and ZEROMV. This patch added checks for these cases, and corrected
the mode decisions.

Borg tests at speed 3 showed:
1. stdhd set: 0.102% PSNR gain and 0.088% SSIM gain.
2. derf set:  0.147% PSNR gain and 0.132% SSIM gain.
No speed change.

Change-Id: I35d17684b89ad4734fb610942d707899146426db

Removed functions:
* vp9_post_proc_down_and_across_mmx
* vp9_mbpost_proc_down_mmx
* vp9_plane_add_noise_mmx

They all have sse2 equivalent.

Change-Id: I59c1fac12b7c96ca4538d455e4400c2b7875feff

Change-Id: I975e5eac16f8b623ff589f0ec072cdaff2183b04

vp9_variance_sse2.c contains a mix of intrinsics and references to
assembly which uses x86inc.asm; it's conditionally included as a result.

Change-Id: I254451483a65881c0b8e18e27bf0c3ddef60c4ec

make bifilter4_coeff[][] uint8_t, no values exceed this range and
they're loaded with vdup_n_u8().

Change-Id: I921983e9edd828d29820e40ac30a7801dbe0fb4f

allocations within vp9_alloc_context_buffers() rely on mi_rows/mi_cols
individually, use those to determine whether to realloc rather than
stride and stride * rows. this fixes a crash with some fuzzed files for
invalid accesses into last_frame_seg_map and above_context.

Change-Id: I7b9f40dcf170d443890f3bd2acd285507943c7d4

proceeding using a corrupt (incompletely decoded) frame reference may
lead to incorrect assumptions about allocation sizes leading to a crash.

Change-Id: I76e74f2e1be127c2e2c7e1174bb3307497dfd23d

This is to keep the same with VP8 denoiser.
If motion magnitude is small,
make denoiser more aggressive.

Change-Id: I942a6e2f2ed9aec6f0c4c1f9e5fa47066cadcc0c

Change-Id: Ifa3c1cc2317c1bc21d1042b9662b35056d1e9ed0

Change-Id: I811526a9ee9f237604f72abe7fc677e39e0f457f

The change is just to keep the condition the same with VP8.

Change-Id: I9662b40996126605945dd853c0cbe8916c1ce578

Change-Id: Ia5e383e2cf18052f6f1eacf8b9495ab8e4d58878

Calling Reset(int) method instead of overloaded operator()(int).
Adding underscore at the end of class member name.

Change-Id: I01934e7bc056d4b594e5d05d693328febd34ac3c

When the first try of denoising turns out to be too much,
we will use a softer filter by adopting an adjustment to
make the result closer to original pixel (as in VP8 denoiser).
The old code made the adjustment in the wrong direction.

Change-Id: I84e28fa9e01eef47c5a37d5a2e6d3d378a06786b

This reverts commit 677fb512

Compiles with 4.6.

Change-Id: I7f87048911b6bc28a61741d95501fa45ee97b819

Change-Id: Ib89107fb824b5fe58afef6841104d5a27b2e0f2d

Use the right return values - vget_low_s64 returns int64x1_t, not
a normal int64_t.

Also make __builtin_prefetch a no-op on MSVC for this file.

Change-Id: I4d2fce01d0ba106b98d3d53b137803119c2c2c08

Change-Id: I6786d25ce4f32b8d8912f2d239a45ca15b310c4b

This commit allows the encoder to store outcomes of single reference
frame modes and compares them to decide if the inter prediction
filter, forward transform, and quantization can be skipped.

The compression performance of speed 3 is down
derf  -0.364%
stdhd -0.198%

For test sequences, the speed 3 runtime is reduced
highway CIF 100 kbps, 51976 ms -> 45033 ms, 13% speed-up
stockholm 720p 1000 kbps, 71826 ms -> 67838 ms, 5.5% speed-up
pedestrian 1080p 2000 kbps, 154924 ms -> 150702 ms, 2.6% speed-up

Change-Id: I5aa26f918d2b4b5197a2c0afa2779319f1c88e44