This commit fixes frame header decoding for superframe index, to
prevent out of boundary memory read triggered by fuzz test
vector. It resolves a chromium security violation issue
crbug.com/376802.

The issue was introduced in the change:

Add VPXD_SET_DECRYPTOR support to the VP9 decoder.
cl-id I88f86c8ff9af34e0b6531028b691921b54c2fc48

where the buffer was read before validation check on index offset
applied.

A test vector is added accordingly.

Change-Id: I41c988e776bbdd1033312a668e03a3dbcf44ca99

The test vector has segment enabled with different quantizer used for
different segments for bot the first frame(key) frame and the rest of
non-key frames.

Change-Id: I7e21122183050ee046219caba483c18cbc34afe7

The test vector is produced to have a single key frame, with segment
map enabled and transmitted. Yet no segment feature is active.

Change-Id: I365d62f00d05c07098b9a76fc8d3a991e427ec1a

Change-Id: I08fe184a04e5435aeac92f1d7cc9733ca52c2783

There was a bug with the decoder that if you started the decoder
with more threads than the first frame had tile columns. Afterwards
tried to decode a frame with more tile columns than the first frame,
the decoder would hang. E.g. run vpxdec --threads=4. The first frame
had two tile columns, then the next key frame had 4 tile columns, the
decoder would hang. If you started with 4 tiles and switched to 2
tiles the decoder would be fine. The issue is that the worker the thread
loop is using is stale.

I added a test vector "vp90-2-14-resize-848x480-1280x720.webm" that
exhibited the bug.

Change-Id: I7bdd47241a52ac0fe1c693a609bc779257e94229

For very large size video image, the scaling calculation may need use
value beyond the range of int. This commit upgrade the value to 64bit
to make sure the calculation do not wrap around INT_MAX.

The change corrected the decoder behavior.

The bug affects only very large resolution video because the scaling
calculation was sufficient for image size smaller than 2^13.

This resolves issue:
https://code.google.com/p/webm/issues/detail?id=750

Change-Id: I2d2ed303ca6482f31f819f3c07d6d3e98ef3adc5

Change-Id: I742b0f26b7d735e75c3653a3e52bd4f4fc198d3d

Vp[89] -> VP[89]

Change-Id: Ib5863444c0c022cc51e2f160ecea92ca157fdf1d

Change-Id: I8cc38fdef6a2a0968af8dfe15e7c2b3c46c531ea

Change-Id: I70240e50927b4379603628f6349bf615f756b5a1

Change-Id: I39286df9fc76e3451f14ce2652d671e11b0c37bc

When showing a previously decoded frame, i.e. when
show_existing_frame=1, the update of the
last_show_frame flag must be disabled.

This is to ensure that the last_show_frame flag
reflects the state of the flag for the immediately
previously decoded frame rather then the value that
was forced to ensure that a previously decoded frame
would be displayed.

This patch also adds a test vector to verify that the
display_existing_frame flag works as expected. Code
for generating the test vector can be found in this
patch:
https://gerrit.chromium.org/gerrit/#/c/68581/

(Bug originally reported by Alexander Voronov
<ru.xalba@gmail.com>).

Change-Id: I731d288fba02088959f7fcc87707137fffc6acf5

From frame 2, the lpf deltas are all cleared for for even frames, and
a set of values are set and used for odd frames. The intention is to
exercise decoding code around lpf delta/update decoding.

Change-Id: Ic9ff1bc2c2a023f4805852f8573398f2ec2249d7

The added vector was encoded with aq mode on, with the intent to
exercise the decode code around segment feature.

Change-Id: Iedcb7261e87d3e11b25ecf031d3a69385271148e

Change-Id: I36f9033492f12fda7f9ce0e7a97a009aab9aab57

3840x2160

vp90-2-08-tile_1x8_frame_parallel.webm
vp90-2-08-tile_1x8.webm

vpxenc crowd_run_2160p50.y4m \
  --codec=vp9 -p 2 \
  --frame-parallel=${fpm} \
  --tile-columns=${tc} \
  --limit=10 \
  --auto-alt-ref=1 \
  --lag-in-frames=5 \
  --target-bitrate=2500

Change-Id: I6dc19b4fca483d03ef9a897a843bf9037d095c8e

The test vector exercises the loopfilter behavior at frame boundary.

blue_sky_1080p25.y4m --good --cpu-used=2 --threads=0 --profile=0
--lag-in-frames=25 --limit=300 --min-q=0 --max-q=63 --cq-level=20
--end-usage=0 --auto-alt-ref=1 -p 2 --kf-max-dist=9999 --kf-min-dist=0
--drop-frame=0 --static-thresh=0 --bias-pct=50 --minsection-pct=0
--maxsection-pct=2000 --arnr-maxframes=7 --arnr-strength=5
--arnr-type=3 --sharpness=0 --undershoot-pct=100 --target-bitrate=6000

Change-Id: Ibd0807395d2fe87f24f81f990369678df3de7c23

Also added a test vector for this issue.

Change-Id: I8a8654f9d33b27afd53c295f1ea44d198550d067

Added the test vector provided by Attila, which caught the bug in
Issue 661 "Decoder produces mismatched outputs with ssse3 enabled
and disabled"

vp90-hantro-stream-001.ivf
size: 320x180; 20 frames

Change-Id: Ic0d2b57ac7596ecb938dd55abc8c706fc2dd6d8f

1920x1080

vp90-2-08-tile-4x4.webm
vp90-2-08-tile-4x4.webm.md5

vpxenc blue_sky_1080p25 --frame-parallel=0 \
--tile-columns=2 --tile-rows=2 \
--lossless=0 --error-resilient=0 \
--min-q=0 --max-q=30 --kf-min-dist=0 --kf-max-dist=9999 \
--cpu-used=0 --codec=vp9 -p 2 --good --fps=25/1 --limit=5

vp90-2-08-tile-4x1.webm
vp90-2-08-tile-4x1.webm.md5

vpxenc blue_sky_1080p25 --frame-parallel=0 \
--tile-columns=0 --tile-rows=2 \
--lossless=0 --error-resilient=0 \
--min-q=0 --max-q=30 --kf-min-dist=0 --kf-max-dist=9999 \
--cpu-used=0 --codec=vp9 -p 2 --good --fps=25/1 --limit=5

Change-Id: I77d0ccf9b721ae5cfa72781fe0d3c534ea43b756

1920x1080

vp90-2-08-tile_1x2_frame_parallel.webm
vp90-2-08-tile_1x2.webm
vp90-2-08-tile_1x4_frame_parallel.webm
vp90-2-08-tile_1x4.webm

vpxenc blue_sky_1080p25.y4m \
  --codec=vp9 -p 2 \
  --frame-parallel=${fpm} \
  --tile-columns=${tc} \
  --limit=10 \
  --auto-alt-ref=1 \
  --lag-in-frames=5 \
  --target-bitrate=2500

Change-Id: Id9c94b722cc553a6865d443a94e8482c78b038bb

vp90-2-07-frame_parallel.webm:
vpxenc stefan_sif.y4m \
  --codec=vp9 -p 2 \
  --frame-parallel=1 \
  --limit=10 \
  --auto-alt-ref=1 \
  --lag-in-frames=5

Change-Id: I7381a69aaaec238b309169a51b34cb6bf29a9c50

Adding appropriate test vector vp90-2-06-bilinear.webm.

Change-Id: Ia3bbf57318e0cc61a1b724fe751e3f9c7e11b337

Modified the resize unit test so that it optionally
writes the encoded bitstream to file. The macro
WRITE_COMPRESSED_STREAM should be set to 1 to enable
output of the test bitstream; it is set to 0 by default.

Change-Id: I7d436b1942f935da97db6d84574a98d379f57fb1

This is a bitstream change but no currently produces videos should
be affected. https://code.google.com/p/webm/issues/detail?id=610

Change-Id: Ic85a6477df6c201cdf7f70f6bd84607b71f4593c

This adds a test vector for 444 color space.

Change-Id: I1e2ac3883211989a062cfafc0e58151b14d294b8

Adding missed parenthesis around boolean expressions. Bitstream is changed.
Regenerating test vectors.

Change-Id: I4cc00b761e9473f92f180a9fc3a0c607f0aaae56

In the rare case were 4x4 interior filtering was called for but no
8x8 or larger filtering takes place, the previous code was skipping
the filtering. This patch fixes the issue by including the interior
mask in the overall mask for the filter application loops.

Change-Id: I4a0b65056c64f97478827c2ff41e0914fc7779d0

if the caller did not abort after an ASSERT failure in Begin()
FillFrame() would segfault.

Change-Id: I2d3f5a0918611bbd081be6f686dea19c56695073

For cases where there's no transform set in bit 0 (the left edge of
the SB) but bit 0 of mask_4x4_int is set (the edge 4 pixels from the
left edge needs filtering), it was incorrectly being skipped before.
This situation only happens on the leftmost edge of the image, as
the edge at column 0 is intentionally skipped since there aren't
pixels to the left to read.

Change-Id: Ib2fbbcb40166e90af31b1a0e13b85b68c226cbd3

Tests resolutions of 8, 10, 16, 18, 32, 34, 64, 66 to exercise the
border conditions, as well as non-SB aligned sizes.

Change-Id: Ie7c2b7860ac3727e23202042f2e86792652912f8

These files can stand in until we get proper syntax vectors. They
should provide some additional assurance against inadvertant
bitstream changes.

Change-Id: I12f6c9a5f054e30df40a7ff1f33145abf7e1d59d

This patch adds column-based tiling. The idea is to make each tile
independently decodable (after reading the common frame header) and
also independendly encodable (minus within-frame cost adjustments in
the RD loop) to speed-up hardware & software en/decoders if they used
multi-threading. Column-based tiling has the added advantage (over
other tiling methods) that it minimizes realtime use-case latency,
since all threads can start encoding data as soon as the first SB-row
worth of data is available to the encoder.

There is some test code that does random tile ordering in the decoder,
to confirm that each tile is indeed independently decodable from other
tiles in the same frame. At tile edges, all contexts assume default
values (i.e. 0, 0 motion vector, no coefficients, DC intra4x4 mode),
and motion vector search and ordering do not cross tiles in the same
frame.
t log

Tile independence is not maintained between frames ATM, i.e. tile 0 of
frame 1 is free to use motion vectors that point into any tile of frame
0. We support 1 (i.e. no tiling), 2 or 4 column-tiles.

The loopfilter crosses tile boundaries. I discussed this briefly with Aki
and he says that's OK. An in-loop loopfilter would need to do some sync
between tile threads, but that shouldn't be a big issue.

Resuls: with tiling disabled, we go up slightly because of improved edge
use in the intra4x4 prediction. With 2 tiles, we lose about ~1% on derf,
~0.35% on HD and ~0.55% on STD/HD. With 4 tiles, we lose another ~1.5%
on derf ~0.77% on HD and ~0.85% on STD/HD. Most of this loss is
concentrated in the low-bitrate end of clips, and most of it is because
of the loss of edges at tile boundaries and the resulting loss of intra
predictors.

TODO:
- more tiles (perhaps allow row-based tiling also, and max. 8 tiles)?
- maybe optionally (for EC purposes), motion vectors themselves
  should not cross tile edges, or we should emulate such borders as
  if they were off-frame, to limit error propagation to within one
  tile only. This doesn't have to be the default behaviour but could
  be an optional bitstream flag.

Change-Id: I5951c3a0742a767b20bc9fb5af685d9892c2c96f

This commit starts to convert the tests to a system where the codec
to be used is provided by a factory object. Currently no tests are
instantiated for VP9 since they all fail for various reasons, but it
was verified that they're called and the correct codec is
instantiated.

Change-Id: Ia7506df2ca3a7651218ba3ca560634f08c9fbdeb

Got 61 test vectors from vp8-test-vectors.git
(http://git.chromium.org/gitweb/?p=webm/vp8-test-vectors.git)

Added decoder test vectors downloading in unit tests. Uploaded
the test vectors and their md5 files to WebM website.
  $ gsutil cp *.* gs://downloads.webmproject.org/test_data/libvpx

Added their sha1sum to the test/test-data.sha1 file.

In unit tests, download the test vectors to LIBVPX_TEST_DATA_PATH.

Test_vector_test goes through the test vectors, decodes them, and
compute the md5 checksums. The checksums are compared with the
expected md5 checksums to tell if the decoder decodes correctly.

Change-Id: Ia1e84f9347ddf1d4a02e056c0fee7d28dccfae15