Inlcudes miscellaneous cleanups, test fixes, and code reorganization
for loop-restoration components.

Change-Id: I5b2e6419234d945e6f4344b22636119b50df4054

This patch forces the vertical filtering for the top and bottom
rows of a processing unit for the Wiener filter to not use border
more than what is set in the WIENER_BORDER_VERT macro.
This macro is currently set at 0 to eliminate line buffer completely,
but it could be increased to 1 or 2 to use limited line buffers
if the coding efficiency is affected too much with a 0 line-buffer.

Also, for the sgr filter we added the option of using overlapping
windows horizonttally and vertically to improve coding efficiency.
The vertical border used is set by the SGRPROJ_BORDER_VERT
macro, while the horizontal border can be set by the
SGRPROJ_BORDER_HORZ macro set at 2, the max needed. Currently we do not
recommend changing SGRPROJ_BORDER_HORZ below 2.

The overall line buffer requirement for LR is twice the max of
WIENER_BORDER_VERT and SGRPROJ_BORDER_VERT.
Currently both are set as 0, eliminating line buffers completely.

Also this patch extends borders consistently before CDEF / LR.

Change-Id: Ie58a98c784a0db547627b9cfcf55f018c30e8e79

Change-Id: I70a4dd8095276eb53050eacca2ee33ba39e7cc74

Change-Id: Ie3912cb012676ccd26d2c0601e21fbb54943b1a0

Sarah Parker authored Sep 08, 2017 and James Zern committed Sep 09, 2017

This addresses visual studio warnings.

Change-Id: Ia57739d8954a81ba5369f05aaf96cdb3144fd0a1

This was causing visual stuido warnings when attempting to write
to seg_iqmatrix and seg_qmatrix using memcpy.

Change-Id: Idf12464c8c90c83af9c2b3ea399d739f7df3b5f9

This addresses a visual studio warning for conversion from 'int64_t'
to 'uint16_t'

Change-Id: I0a8c67938c1b50d4eb33aa43cf368a21c60f49a8

This patch allows ncobmc-adapt-weight to replace warped-causal,
and also remove transmitting ncobmc_mode_prob in the frame header.

Change-Id: I2713b3abbd28eef8b203637e432eca6a782e3b9a

Change-Id: Id3453fcf61fced97810d68cc7ca285bf8bca7881

Change-Id: I949558a3fc716e717e3d0839a6a1ae4684ca7125

Intrabc is counted as inter mode, but it doesn't support var-tx yet.
This patch skips tx partition info update for intrabc blocks. It
resolves a decoder crash issue.

Change-Id: I97e413e4a837ba6510e110347f5aa6e5fd14dcc5

BUG=aomedia:685

Change-Id: I592016faeb4645cf43c599e0e2e11127e2a17c4e

Change-Id: Ia84ea2f4e855671f5a29058cc7a6b5ba9e6bfef9

Tom Finegan authored Sep 07, 2017 and James Zern committed Sep 08, 2017

Change-Id: If4801b24e8711ab1e78a86b6dbb504be59042200

Tom Finegan authored Sep 07, 2017 and James Zern committed Sep 08, 2017

The CMake compiler test functions are built atop CMake's try_compile()
function. try_compile() ignores configuration specific link flags in
favor of the generic linker flag variable (CMAKE_EXE_LINKER_FLAGS is
used instead of CMAKE_EXE_LINKER_FLAGS_<CONFIG>). This patch applies
the flags cached in AOM_EXE_LINKER_FLAGS to CMAKE_EXE_LINKER_FLAGS
during C and C++ compiler tests.

Change-Id: I831244d114a8195527bcb865537494a98cabc170

Tom Finegan authored Sep 07, 2017 and James Zern committed Sep 08, 2017

When building without SSE4 support some compilers complain about
an unused variable. Typically gcc's with a major version of 4.
Guard the declaration of the offending var within a
HAVE_SSE4_1 block to avoid the problem.

Change-Id: I4e4deb46014c97f3157f3b6c2376e1b034a51b62

Do not refer to above MB when on SB boundary. This helps to
reduce line buffer.

Keyframe coding gain on the screen_content testset drops from
1.28% to 1.20%.

Change-Id: I4b06fd137d6c9ea68d618035381f09d0746ba9e8

Encoder may do multiple rounds of transform RD search on the same
residue signals. Save the RD search results so that encoder can
fetch the results and terminate early if the residue signals have
been seen before.

The hash functions are ported from the hash_me experiment.

Test results show the encoder speed is increased by 10% on
average (ranging from 0 to 30%) with all default experiments on.

Change-Id: I47dd63549f67cb43d3d700f6a76ce0992b1ccd0d

A lot of local functions have av1_ prefix due to legacy reasons.
Make them static and remove their prefix.

Change-Id: I069a3a1af583d5d307cdc56819901d1f9e004d07

Use macro SCALE_SUBPEL_BITS rather than hard-coded 4 for old
warping code that is used for non-affine global models.

Change-Id: I10ee7b29101cd79e77a4d29e69d67497fda4e967

Newer gcc's output logical-op warnings when and'ing together
a duplicate condition. In this case it's because a constant
was set to an enum value that was also checked locally.

Change-Id: Iada9571d3e59ec9b75aa446fd6690587092af161

Save bits by not encoding global motion data when there's no
previous frame to use as a reference. Use the same logic
as for using the previous frame's motion vectors - ie, we don't
read global motion information when the current frame is
intra-only or when using error-resilient mode.

This also fixes an undefined behaviour / segfault bug - see
https://bugs.chromium.org/p/aomedia/issues/detail?id=731 for
details

BUG=aomedia:731

Change-Id: Icca90a1bccafd06de8a4056ca5353318fce416cb

Change-Id: I43e92e3f3799857e928543434d049b4155b64d7d

Baseline + parallel_deblocking:

- Passed unit tests *SSE2/Loop8Test6*, *AVX2/Loop8Test6*.
- 1080p, 25 frames, profile=0, encoding/decoding, output match.
- Decoder frame rate increases from 54.15 to 65.84.

Change-Id: I55938c94961066594f4b9080192c7268c19d9bf9

Change-Id: I0e9df3719e5f9a55e1386afe44851d1707e2e01b

The callsites to decode_block and decode_partition in the body of
decode_partition are rather noisy with code that is enabled or
disabled by config flags, so it's hard to see what is fundamentally a
simple recursion.

This patch moves all that cruft into a pair of locally defined
preprocessor macros (DEC_BLOCK and DEC_PARTITION) to simplify
things.

There's no diff in disassembled code when compiling before and after
the patch.

Change-Id: Ie08db263a9874e2c566c758e8f3a51d131b1b80b

This patch forces the vertical filtering for the top and bottom
rows of a processing unit for the Wiener filter to be 5-tap.
The 5-taps are derived from the primary 7-tap fitler by forcing
the taps at the end to be zero, and absorbing their weights into
the other taps to maintain normalization.
This will effectively reduce the line buffer size for luma Wiener
filter to 4 (from 6).

Change-Id: I5e21b58369777eabf553a8987387d112f98a5598

As of patch https://aomedia-review.googlesource.com/c/20220 ,
it is no longer always the case that tile_rows == (1 << log2_tile_rows)
and tile_cols == (1 << log2_tile_cols). This exposed an inconsistency
in the tile-group decoding:

When reading the first tile group in a frame, we read
(log2_tile_rows + log2_tile_cols) bits each for the tile group
start and length, but when reading later tile groups we read
get_msb(tile_rows * tile_cols) bits. But now there are edge cases
where those values differ! Since the encoder always uses
(log2_tile_rows + log2_tile_cols) bits, this leads to mis-parsing
the bitstream.

Fix this by moving the decode logic to one function, which always
reads (log2_tile_rows + log2_tile_cols) bits. As a bonus, this gives
us one place to check other invariants, eg. that tile groups can't
run off the end of the frame.

Change-Id: I83b24314526b6055300b70b0f1cdce038e6b23dc

Change-Id: Id606669875ef1dcb163ff7dec619dae6110d9047

This requires making a temporary copy of the functions in
binary_codes_writer/reader to take in the aom_write_bit_buffer type.

Change-Id: Idb60b29cff69b45224535c6e6a4079a34a2c6871

Fix the bug forcing encoder/decoder to transmit/decoder all
mbmi info for a superblock first.

Change-Id: I623217655b043fc90adbcc13e4cf2a4a845084ab

Change-Id: If60c97131bed58ce7c6d3875f9a088df27b9bae4

In the old code, the function of allocate_gf_group_bits() in
firstpass.c handles both GF group structure design as well as the
corresponding bit allocation for each coded frame. This CL separates
the two functionalities.

It has been verified that no coding performance has been incurred by
this CL.

Change-Id: I544b4fcdc8cb37ec810373f1f820dbd2e400c39b

Update cdf after each superblock being finalized, and re-compute
symbol costs. Affected symbols include prediction parameters and
tx type.
BDRate: -0.119% lowres, -0.092% midres

Change-Id: I724900ac04f725910ed7055a18c2f6a139469b52

Merge the tile context models at the end of frame coding.

Change-Id: I0a71c49c448cfbd71fb0bd15ca7b1f6097c56529

BUG=aomedia:719

Change-Id: Id47c2c68cb61e142ce62fee5e97f7c474b3dc273

Change-Id: Iff1a7e35c439f0baa7240eef75fcb5da46d06a4c

Change-Id: I3de1c933ee0fa90e9c0d52e6cbe4bc8bf5482a73

Change-Id: I7393596d98f11aa53ba4b9e329386b5168b3e086

Add AOM_EXTRA_<TYPE>_FLAGS variables. Currently AS, C, CXX, and
LINKER are the available type values. These variables are added
to argument strings for each type as the last step in generation
to ensure that user flags have precedence over flags added by
the AOM build itself.

Change-Id: I8e01682ee6c7dad2253e01c446a5ced7d06596c7