Previously the transform type sets used for 16x32/32x16 transform
sizes corresponded to 16x16 transforms. Now they correspond to
32x32 transforms with a reduced available set.

BDRATE impact:
0.05% worse for lowres, 0.13% worse for midres

Change-Id: I01d405d497c13837f5404f9e0caf8c571f3b26cf

BUG=https://bugs.chromium.org/p/aomedia/issues/detail?id=76

Change-Id: I14dd6704450a06898ca1bc5c0f4f9b21316c4df1

- Properly respond to detection of i386 and x86 as the CPU.
- Restrict -Wshadow to C files: it silences a large number of
  warnings caused by test code.

BUG=https://bugs.chromium.org/p/aomedia/issues/detail?id=76

Change-Id: Ia15ae8ed0fa90aec587818bda778a638c811168d

Change-Id: Idc24c2dc8015d14ea0c36449955a37614b750beb

increases the translational part of global motion models
to the same 1/64 pel precision which is used by the warp filter.

Change-Id: I7267b18b123662416bc5c8095186b6a5eef2f7f1

The full-pixel checking is conducted in update_filter_type_count
Change-Id: I583ec4b1f3ac29d124098819aee257a8f1007ed4

Removing av1_is_interp_needed() function from it.
Use has_subpel_mv_component() only.

Change-Id: I98b674b28c276a09bb94f24d66360d24377066a8

Change-Id: Ib3797b7e0ea6050225a56464042deaa856f6a72f

Change-Id: I2c05ce6a96cc21826fc53aeacce91d9637d3b178

Change-Id: Ie7edfbcdd7f5f2cc3aa3aa6c2575d081616da6b6

Here we create a default set for the ext-tx experiment which
includes:

Intra: {DCT, ADST} X {DCT, ADST} + IDTX (identity in both directions)
Inter: DCT_DCT + IDTX

This set is used if indicated in the uncompressed header.

Change-Id: Ic907ada12b4e0c98cabb7c9650ab99942d0c0859

By rearranging the order of operations, we can ensure that all
intermediate values fit into 32 bits. This will help when we
vectorize the self-guided filter.

Results in the noise range.

Change-Id: Ic0c73613882bd103c4e8e57a0155b3132672ae04

Remove code related to convolve_round experiment out of
av1_convolve()

The experimental code is now implemented in av1_convolve_2d().

Change-Id: I409a390876e9d19e6ee42150f0ac20ff108244a9

Some of the conditions in 'else if' blocks were redundant. Getting
rid of them should improve speed.

Change-Id: Ib5a994c54c4913ca68c7d982abddc85c0b36db0b

Signed-integer overflow is undefined behaviour in C.

Change-Id: Ib2d7f405adacc668264a5d77aba00ca85ecbb0f7

De-duplicate implementations of 'range_check', and use precomputed pixel value
boundaries instead of computing the log2 of each pixel value.
Overall speedup: encoding with enabled-range checks goes 3.3 times
faster.

Change-Id: I2f7c97cac4ad479b07898fb69c0131e3c7e0bf98

Use the top-left 4x4 luma block's coding information for chroma
component encoding at size 4x4.

Change-Id: I4bcfbc2bf8b71f7fc30094553503468460a56f9b

The result is identical to enabling both deringing and CLPF

Change-Id: I71db5ba9e21fcaf11ad87e94841eaf80be58c0a8

The ATTRIBUTE_PACKED caused something to break in MB_MODE_INFO.
boundary_info got corrupted and moving InterpFilter around in
MB_MODE_INFO could cause a segfault or the problems to go away (for
instance by moving InterpFilter to the end of the struct).  Perhaps
something in MB_MODE_INFO is sensitive to alignment.  This should be
investigated further and get fixed or documented, but for now remove
the attribute.

Change-Id: Ic4326ab98fe784d9b7d2b47d8f086b8429cedee9

Change-Id: I3fa9d24f2a61dd1b4a234fb1cfea679af06ebdb5

Remove redundant table access.

Change-Id: Ib45f9b0bafe2a0c57449e787d60e70d35f19f6c7

x86 and x86_64 support is now working for most platforms. Remove the
automatic fallback to a generic unoptimized build, and require users
to explicitly request generic to build a generic library and tools.

BUG=https://bugs.chromium.org/p/aomedia/issues/detail?id=76

Change-Id: I7b36394dc28aa9b729d2b2504146d02f017c9a0b

When run with --enable-ec_adapt, forward updates to the tx_size_prob
 array will not be made.

objective-1-fast:

master-ec_adapt-e568e6f0@2017-02-17T15:30:27.109Z ->
 tx_size-ec_adapt@2017-02-17T15:31:49.504Z

   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.0113 | -0.0112 | -0.0113 |  -0.0114 | -0.0114 | -0.0114 |    -0.0112

master-ec_adapt-robust-e568e6f0@2017-02-17T20:29:45.315Z ->
 tx_size-ec_adapt-robust@2017-02-17T20:33:06.405Z

   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.0244 | -0.0243 | -0.0243 |  -0.0245 | -0.0245 | -0.0245 |    -0.0243

Change-Id: I5873c5d14e3f3868997775f85fa743d2d55d81b7

When ext-intra or filter-intra is enabled, the transform type was inferred from
prediction mode as in VP9. Should use mbmi->tx_type instead.

Change-Id: I6a8e356285a8461d67e69d38df40541d34bade04

We use a single enum instead of multiple #defines.
- Ensures better type checking
- Enum values are generated implicitly, and hard-coded #defines are not
required.
- We use ATTRIBUTE_PACKED to indicate that the enum should still use the
smallest integral type.

Change-Id: I7532428da31744d3441b363bd932a7f233ee7ab5

Replace obselete size "4" with correct one "tile_size_bytes". This
fixes a decoder crash.

BUG=aomedia:352

Change-Id: I5b1c606a4d5be946acc68b3cfe0d0603a5591c44

Replaces division with multiplication in self-guided
filter.

The guided filter requires computation of:
n^2.s^2/(n^2.s^2 + n^2.e).
This is now implemented by computation of n^2.s^2/n^2.e followed
by using a lookup table for the function f(x) = x/(x+1).
To compute n^2.s^2/n^2.e, we use an integer multiplication based
implementation which becomes feasible since n^2.e can only
take a few values and their corresponding multipliers can be
pre-computed.
There is also another divison by n, that is also integerized.

Change-Id: Id7b81bbafead0b8f04a1853ec69b9dec423bb66a

Adds functinoality for least-squares, RANSAC as well as encoding and
decoding with new constrained homographies that warp blocks to horizontal
and/or vertical trapezoids. This is for future experimentation. None
of the models are actually enabled in the code.

Change-Id: I1936018c6b11587d6fd83c3a2c63548cb641b33f

Change-Id: I7909756e6a7d2da817b3354d6a8cb341cdc5bf98

This commit produces a bit-exact bitstream because prior calls to code
 the tx_size using aom_write_tree() would automatically convert the
 aom_tree_index on the fly to a CDF.
The tx_size_cdf[]'s are now computed once per frame.
Based on recent tests, this patch reduces decode time by 4%.

Change-Id: I920dbf036b8a9574ea6e65ae5f9c43eef1c3f864

Which was a workaround to balance the quality between
luma and chroma when PVQ is enabled, by
1) skipping PVQ DC RDO for chroma and
2) using smaller lambda value (x0.5) for intra prediction of chroma,
intended to spend more bits and thus improve quality for chroma.

The change in metric for 5 frames of objective-1-fast is:
PSNR  PSNR-HVS  SSIM  CIEDE-2000  PSNR Cb PSNR Cr  MS-SSIM
-1.45 -1.54     -1.46 4.62        13.71   14.87     -1.47

For still images (subset1)
PSNR  PSNR-HVS  SSIM  CIEDE-2000  PSNR Cb PSNR Cr  MS-SSIM VMAF
-1.52 -1.54     -1.60 2.56        11.92   12.17     -1.52  -1.66

Change-Id: I31c4d7a7de2939f6004c112649f7303f0e944156

Low-latency:
masterfix-ll-dering+clpf@2017-02-14T02:31:30.023Z -> masterfix-ll-dering-nosecond@2017-02-16T21:30:38.937Z
   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.1151 | -0.2440 | -0.1287 |  -0.0454 | -0.1130 | -0.0632 |    -0.1206

High-latency:
masterfix-dering-clpf@2017-02-16T21:28:58.511Z -> dering-nosecond+clpf@2017-02-15T23:46:08.501Z
   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.0340 | -0.0883 | -0.0130 |  -0.0104 | -0.0026 |  0.0245 |     0.0215

Change-Id: I3e8f34b1971d3ec280c5695a6fcfb7d7b270b2d9

So far the overall performance drop by
lowres: 0.027%
midres: 0.049%
hdres: roughly 0.07%

Change-Id: If3dc7619d91cb97b6d494b33f36112af45efcd8f

Change-Id: I8a23fcf0839c2c2b1d638c3e0fc9d4e5a09f1269

This commit implements a new rate control system which was ported from
Daala's rate control system (which was based off of Theora's rate
control system) to the encoder. Its aim is to address the weaknesses of
the current rate control system and eventually serve as its replacement.

This is still work in progress. So far this commit only implements
support for single pass encoding for constant quality and bitrate
targeting.
Additional commits which implement two-pass support and the rest to
match the feature set of the current rate control system are coming
soon.

The rate control system hasn't been fully tuned (and will likely need
additional tuning as the encoder's development progresses) and does
contain some "hacks", to convert the quality to a quantizer value
(empirical) and to tweak the quantizer in between golden frames (which
uses some code from the current rate control system as well as code
which attempts to model what that code does), the latter of which is
only in the constant quality codepath.

Bitrate targeting works much better than the current rate control
system's targeting and will actually closely match the rate specified by
the user without the current rate control system's bursty behaviour.

Change-Id: I588fbfd2e80a3d21ce7176903115d6a96ef1700a

Since token coding has changed the relationship
between EOB values and non-zero values, coeff costs
and trellis quant need to change to be more accurate.

Change-Id: I27ef400e8290db4c5faa9c21a253575eea7955c4

When the "dy" variable is between -1 and 0 (base2 == -1)
we should interpolate between the upper-left(-1,-1) and left(-1,0)
samples, instead of just taking the left sample.

Coding gain improves a bit with this change.

BUG=aomedia:357

Change-Id: If520f71b22115e79120499c21f8d6925c1fa24be

This fixes a enc/dec mismatch.
BUG=aomedia:354

Change-Id: I727c27a412c21695eb3c084f8dd688163615efe3

Working with ninja, make, and xcode generators using the
following args at cmake generation time:
-DAOM_TARGET_CPU=x86 -DCMAKE_OSX_ARCHITECTURES=i386 -DCONFIG_PIC=1

BUG=https://bugs.chromium.org/p/aomedia/issues/detail?id=76

Change-Id: I830db21f7458417cb8048818e5681a78698a6068

Rename the aom_read_tree_bits() function to match aom_read_tree_as_cdf().

Change-Id: I779f544615b1c19bb1f781c1f7a0f59dda7bd466