BUG=aomedia:310

Change-Id: I8e1a1c6d59e3d14ba132d2bbf4e203da26538bde

Resolve the broken coding pipeline in ext-inter experiment when
cb4x4 mode is enabled. Turn off rectangular inter-intra mode.
This needs some more work to hook up. Given that it gives fairly
limited coding performance gains, disable it for the moment.

BUG=aomedia:309

Change-Id: I9b406df6183f75697bfd4eed5125a6e9436d84b0

BUG=aomedia:276

Change-Id: I7a8867341a2434c8044d20e0ac5128bc1b1d8cfc

BUG=https://bugs.chromium.org/p/aomedia/issues/detail?id=76

Change-Id: I713c8743a0a4fde4bfc860a5c2d73d53dd9645d9

Change-Id: Ieb2922c3df4ef4f8514b8a6df6f9a8fc45ef3cf4

Change-Id: I148d60d56599a238c60c429572a25cbddbe5191d

This follows the naming for the other frame types, and allows libaom
to be compiled against other libraries that also #define NONE.

Change-Id: Ic2e2814587bbc5ea67385a9af775396d29b7dde0

* The restriction on the parameter 'delta' was too strict, so we
  loosen it (delta only ever gets multiplied by -4, ... , 4,
  whereas beta gets multiplied by -7, ..., 7)
* Correct a comment about the border clamping
* Fix an issue with the test case

Change-Id: I30e55203455ba6e419b5a8b646151a6d1fd5cc3b

This commit adds a new experiment, Daala's distortion function,
which is designed to better approximate perceptual distortion
in 8x8 pixel blocks.

This experiment is expected to work best with PVQ.

It measures the variance of overlapped 4x4 regions in the 8x8 area,
then uses these variances to scale the MSE of weighted frequency domain
distortion of 8x8 block.

Since AV1 calculates distortion in blocks as small as 4x4, it is not possible to
directly replace the existing distortion functions of AV1,
such as dist_block() and block_rd_txf().
Hence, there has been substantial changes in order to apply
Daala's 8x8 distortion function.
The daala distortion function is applied
after all 4x4 tx blocks in a 8x8 block are encoded (during RDO),
as in below two cases:
1) intra/inter sub8x8 predictions and
2) 4x4 transform with prediction size >= 8.

To enable this experiment, add '--enable-daala-dist' with configure.

TODO: Significant tuning of parameters is required since the function has
originally came from Daala thus most parameters would not work
correctly outside Daala.
The fact that chroma distortion is added to the distortion of AV1's RDO is
also critical since Daala's distortion function is applied to luma only
and chroma continues to use MSE.

Change-Id: If35fdd3aec7efe401f351ba1c99891ad57a3d957

This commit resolves an enc/dec mismatch issue when both filter-intra
and cb4x4 modes are enabled.

BUG=aomedia:253

Change-Id: I4026d93c00a819f2ce69aedba9d34a774319acbf

BUG=aomedia:142

Change-Id: Id21dd2d19e1e46a9225cd5f8f8b0705ae178118c

Change-Id: I2a20b3eb8020e3e3592d284737acd5da13bad103

hui su authored Jan 20, 2017 and James Zern committed Jan 21, 2017

This is due to encoder speed optimization. Bitstream specs and
decoder behaviors have not been changed.

Change-Id: I4d601d6013f769e69dcec15f2a376e43cc76b197

This commit enables the adaptive scan order system support
rectangular trnasform block sizes. It resolves the coding failure
when rect-tx or var-tx are enabled.

BUG=aomedia:143

Change-Id: Ic565284e811e3f7e0ebf2e08fb3748257ce8a049

Fix an encoding failure issue when var-tx is enabled, while ext-tx
and rect-tx are disabled. This doesn't change coding statistics
when all are enabled.

Change-Id: I4b32387a0a1497380980f8087832aaf6467cdcbe

This commit makes ext-tx and rect-tx experiments supported in the
cb4x4 mode. It resolves an enc/dec mismatch issue when all the
transform experiments are enabled.

The coding gains are
        ext-tx + rect-tx   cb4x4    vartx     total
lowres      4.0%           2.3%      0.5%     6.9%

The encoding speed is about the same when cb4x4 and vartx are
further enabled.

BUG=aomedia:139

Change-Id: I3fdabc6d5de23ceb78ac0751a9bf7332ebc0a3ac

assume __clang_major__==0 has the latest version of
_mm256_broadcastsi128_si256. fixes builds with custom clang toolchains.

cherry-picked from libvpx:
33aef48f2 vpx_subpixel_8t_intrin_avx2: tolerate unversioned clang

BUG=b/30970831

Change-Id: I90becd56278e4716bd46e2ba9d910af977e8dfa6

BUG=aomedia:132

Change-Id: I5c3214ddbc97576a2e90a070f2bdccc15be50d65

Change-Id: I54a0fbaeb59de0907a17b73dab4170cf62b4fd8d

Also adds --framestats=file.csv to aomdec that prints a CSV file with
frame size and frame QP.

Change-Id: I3b70c4b3df35d0b97bdd83cfc4631f096573b4a2

Change-Id: Ied58b6e4a15259cf24e3ee490c042767f4a48f16

Change-Id: Ia2fe7cb7f0d0a98d3050c1db059ec04e3735e1ec

Account for the additional block sizes in these tables.

Change-Id: Iae940f28671714caaf32432940752958ef66f6d5

Properly determine and use horizontal and vertical masks
for loop filtering when rectangular transforms are used.

Fixes an intermittent mismatch issue and improves coding
efficiency.

BDRATE results for ext-tx + rect-tx:
lowres: -3.739% (up from -3.443%)
midres: -3.366% (up from -3.006%)

Change-Id: If26fa14261f3893662eb1245f0b876d68513247c

Move declaration of filter_params_x/y outside of if/else block

Change-Id: I4f908872b7ff85b440a12a535d939a3c137aaab5

By turning on CONVOLVE_POST_ROUNDING, in the compound inter
prediction mode, FILTER_BITS rounding is moved after the summation
of two predictions.

Note that the post rounding is only applied on non-sub8x8 block

       PSNR     BDRate
lowres -0.808%  -0.673%

Change-Id: Ib91304e6122c24d832a582ab9f5757d33eac876c

BUG=aomedia:71

Change-Id: Ie6fafd716fc0419cd170771d51eafa9a915cd790

Change-Id: I7bbd3c62341ede45628641766b8683b77f3a7efb

Change-Id: I522dfe77cbe0ea4833d11e25386586d7312c463f

Change-Id: Idd7926f0539a0bd039828e5882392fbfb024e531

Instead of returning skip, av1_pvq_encode_helper and od_pvq_encode now
return ac_dc_coded. This gives more information on whether the DC part
or the AC part was skipped.

Although it is possible to obtain ac_dc_coded from the pvq_info struct,
this struct is not always used, in which case the information was lost.

This change does not impact the bitstream.

Change-Id: Ie303de915f74e8da384f822332eb1aa27f677bd3

Change-Id: I4b53dab674390496d8fe7299970c5fb327b5a7be

Change-Id: I61433d0c0bbab9b7cf74a405cbedd60965318888

Change-Id: Id01c785ad48134075c4f6643233413564f0b8fbc

Change-Id: I71c9bedfae2304c201fe6621a20c03f4e26a85cf

This commit fixes an enc/dec mismatch issue in ext-partition-type
in the cb4x4 mode.

BUG=aomedia:137

Change-Id: I19f538a967a6059a40b1668eed076bc315b46149

Fix the intra block coding order when both ext-partition-type and
cb4x4 modes are turned on.

Change-Id: Iaaaf4742c53c4778526974f9d1dfdaed6ca3ce3c

This commit resolves the coding pipeline breakage when ext-partition
and cb4x4 are both enabled.

BUG=aomedia:138

Change-Id: Ic17da68af80d7a66565b0e1c69b895be27282a9a

If _d == 0 we are already off to the UB races due to out of bounds
access in OD_DIVU_SMALL_CONSTS.

Change-Id: I55a76c51483885bbb38667f14836be9830e130a8

Use the right number to free up the allocated memory space in
context space.

Change-Id: Ic2950c133d6234b9a4216283a6f4a1dea13128d1