This avoids the rounding errors due to the right shift of the
negative numbers that cause the reconstruction coefficient has
higher distortion than the source coefficient.

BUG=aomedia:599

Change-Id: I11ed86bf1d41164dda4398545334a7b4e8e10513

Change-Id: Iab2f7ca3c66f8f62b66100e61ff0f6e7376670ae

The highbd_clip_pixel_add() function is generalized to be used in
the regular 8 bit path. Move its defintions outside the highbd
experimental flag.

This resolves the comiler warning in unit tests when high bit-depth
is turned off.

Change-Id: I90a744adb2381c9bf8476aa2a2bd0c87d9afdf57

This was hard-coding the assumption that the block size for the
smallest TX size was also the smallest block size. This is no
longer true since fe67ed6a landed.

As a result, for TX blocks that overlapped the frame edge, it was
only measuring distortion on the upper-left 2x2 part of each 4x4
sub-block, causing the encoder to prefer larger transforms which
cause such overlap and avoid transforms which do not, causing a
regression.

This patch uses the appropriate conversion table, which fixes the
regression.

BUG=aomedia:593

Change-Id: Id253cf0f3a5252378e3f340b8350120639ff5c88

BUG=aomedia:597
Change-Id: I6ff109fac8351a98a672259532520e062c1e6497

David Barker authored Jun 08, 2017 and James Zern committed Jun 09, 2017

The Windows calling convention pushes any __m128i type arguments
after the 3rd (4th on x86-64) onto the stack. But on x86,
stack-allocated arguments are not guaranteed to be aligned to
a multiple of their natural alignment, leading to compile errors.

We fix this by making the functions which take >3 __m128i arguments
instead take pointers. Since the functions are marked INLINE, the
extra memory operations should optimize out.

BUG=aomedia:587

Change-Id: I0cb2831fd12aded6f2821c037365386e6183ba5c

Includes a test case based on the warp filter tests

Change-Id: I9abea53a088f68bb8a928ebd7cb96b3266a63c13

Bug introduced in change: I34cdeaed2461ed7942364147cef10d7d21e3779c

BUG=aomedia:591

Change-Id: I49b9edd2bf5a482b5afea5d83d56e04a0086f797

* Reduce bit widths of intermediate values where possible
* Change ROUND_POWER_OF_TWO_SIGNED to ROUND_POWER_OF_TWO
  in av1(_highbd)_convolve_2d
* Apply offsetting and bounds checking, to match the intended
  hardware implementation
* Separate the implementations of av1(_highbd)_convolve_2d
  into compound-round and non-compound-round cases. This is because
  there are now a significant number of differences between the
  functions.

Overall, this is expected to affect the bitstream and encoder output
when convolve-round alone is enabled, but *not* when compound-round
is enabled.

Change-Id: I8c21e0645fd11f64c59552885f87f4a5dd40ccf7

The 'ref' member of ConvolveParams currently serves two purposes:
* To indicate which component of a compound we're currently predicting,
  eg. for fetching interpolation filters with dual-filter enabled.
* To determine whether we should average into the destination buffer.

But there are two cases where we want to separate these out:
* In joint_motion_search, we want to try combining a fixed second
  prediction with various first predictions.
* When searching masked interinter compounds, we want to predict
  each component separately then try different combinations.

In these cases, we set 'ref' to 0 and use temporary variables to
make sure we use the correct interpolation filters. But this is
quite fragile.

This patch separates out the two uses into separate members.
This allows us to remove some temporary variables, but more
importantly gives easy fixes to two bugs in
build_inter_predictors_single_buf (used by rdopt):

* We previously set ref=0 but didn't fix up the interpolation filters
* For ZERO_ZEROMV modes, the second component would accidentally
  average into the (uninitialized!) second prediction buffer

BUG=aomedia:577
BUG=aomedia:584
BUG=aomedia:595

Change-Id: Ibc31d1ac701a029ea5efaa1197dd402bc4b7af1e

This has been found to be better than the original version in both ways:
(1) Better compression: lowres -0.229, midres -0.147
(2) Faster too in my quick test over 5 different clips with 30
frames: 2.7% to 10.5% faster.

Change-Id: I4d46e0915d6e4b8e7bfc03d0c8b88cbe3351ca20

2x2 transforms are now hidden behind the CHROMA_2X2 macro,
not the CB4X4 macro.

Change-Id: I5d73c679fba486ccda98fa8dbb804a3902df6c8d

Minor updates in test/tools_common.sh to enable use of
test/examples.sh with CMake make builds while continuing
to support configure builds.

BUG=aomedia:76,aomedia:589

Change-Id: I841aef3b61a0c9baa8ad7356fc5b51ffb0902907

Sebastien Alaiwan authored Jun 08, 2017 and James Zern committed Jun 08, 2017

Visual Studio doesn't support it.

Change-Id: Ie7bd21e6d9a8a30b7b89e9fca6b82dd2aa406a89

BD-RATE PSNR twitch-1 1st KF: -0.1334
BD-RATE PSNR wikipedia 1st KF: -8.8791

Change-Id: I882af1d317ebdd3017c23fb884fc624282a2a55f

This option is obsolete and confusing now that the LBD coding path
can be disabled at build time (--disable-lowbitdepth).
Removing it makes the encoder/decoder behaviours symmetric, and
prevents the "--test-16bit-internal --test-decode=fatal" mismatch.

BUG=aomedia:528

Change-Id: Ia2d9857629b789b11d37fc75433b2cecc27d6642

This will fix Assertion in get_txb_ctx() for invalid sign value

After fixing this bug, we have around 0.2% gain on lowres.

With cb4x4 on only, lv_map provides 1.538% gain on lowres.

With cb4x4 + chroma_sub8x8, lv_map provides 1.468% gain on lowres.

Change-Id: I06f996ec5af0d5e79e377a3dc8c012862fc4b9c7

For a tx size RD search with partition size >= 8x8 and tx size < 8x8,
daala-dist function is applied to the whole partition after all tx blocks are encoded
instead of each 8x8 sub block of the partition.

Change-Id: I27d9e2960aa641f550096e32ebcdf8dfb4de79a6

Change-Id: Id926529b11f859aebed43fdcfcdedba9ce26b1a7

Cleanup related wrappers and unit-tests.

Change-Id: I2d37a8c80de63dbeaef584e3d5fa842c0b2ee6db

This reverts commit bf3813a1.

Reason for revert: feature not ready.
Incompatible with lossless under some circumstances,
causes the following assertion failure:
Assertion `(!is_compound) == (cm->reference_mode == SINGLE_REFERENCE)' failed

BUG=aomedia:575

Change-Id: I63a2b38ce3b7cb50108ac559cca0768b4579c9ae

Change-Id: If00a0bdf239b2c9e355cffd2e472708acb189f16

This unifies the codepath for high-bitdepth transforms and deletes
all calls to the old deprecated versions. This required reworking
the way 1d configurations are combined in order to support rectangular
transforms.

There is one remaining codepath that calls the deprecated 4x4 hbd
transform from encoder/encodemb.c. I need to take a closer look
at what is happening there and will leave that for a followup
since this change has already gotten so large.

lowres 10 bit: -0.035%
lowres 12 bit: 0.021%

BUG=aomedia:524

Change-Id: I34cdeaed2461ed7942364147cef10d7d21e3779c

Change-Id: I0512000554ef74d397332e5ed135fe20e2c4a37e

- Add unit test for av1_block_error.
- Fix av1_dist_block logic for calling av1_block_error.

Change-Id: Id8a47ee113417360a29fc2334d9ca72b5793e2d7

Change-Id: I2e83419aecf6464fa76f091015597207a12bd478

Saves some bits here and there, no effect on reconstruction.
-0.13 BD-RATE PSNR on wikipedia_420

Change-Id: Idfa137fe61b828fd99eaecda84afa2eaaf893d71

The WithParamTest constructor needs to be called first, as the
EncoderTest constructor is called with 'GET_PARAM'.

BUG=aomedia:580

Change-Id: I17f3c02c0a0eec103fb3505301b5eb6dfdef851c

Fail at configure time when required features are not
present. Currently requires only necessary x86 object
formats and the presence of the -Ox opt mode arg.

BUG=aomedia:76

Change-Id: Idc372e8ed121a600e87c46c0d29d5322cfceaec8

Change-Id: I3ce09c03d4306c1f2a9c70aa1e82126493bc9406

Change-Id: I403f7ab5aa27894d56d9efdbb52c7ded223dd11a

Change-Id: I4e9e6dc6c86de24c6a33d7078c29a853e142c062

objective-1-fast 1st KF: -0.07 BDRATE-PSNR
twitch-1 1st KF: -0.04 BDRATE-PSNR

Change-Id: I089900514c40f3b8b77708dac2c8bfbce2f540ff

objective-1-fast 1st KF: -0.04 BDRATE-PSNR
twitch-1 1st KF: -0.04 BDRATE-PSNR

Change-Id: I74e8e43278a3d228f9b0a9af014e69f80aa90a0f

- Adds the template file to the target enabling easy
  edit via IDE editor.
- Drop a generated file comment at the beginning of
  the generated doxyfile in hopes of informing users it
  should not be modified.

BUG=aomedia:76,aomedia:559

Change-Id: Id4e2762a5dae145927e7b7f1f6689744150ba090

* Make intermediate arrays in av1(_highbd)_warp_affine_c signed,
  to avoid integer overflow when multiplying an 'unsigned int'
  by a negative 'int' value.

* Pad out arrays in masked_variance_test.cc so that the array
  stride is a multiple of 16 bytes.
  This fixes some UBSan errors in masked_variance_intrin_ssse3.c
  related to unaligned loads of 32-bit values.

BUG=aomedia:572

Change-Id: I0cf786c94870ff128c883bed8e900b0686afc3f7

Put the test cases for av1_warp_affine_ssse3 behind a
"#if HAVE_SSSE3" condition

BUG=aomedia:582

Change-Id: Ide55dc1cf6a1d1b0c37db6d17488ac3f8e113888

Earlier, intra prediction for rectangular blocks was performed by
running two steps of prediction on square sub-blocks.

With this experiment, we do proper intra prediction for rectangular
blocks. This ensures that we make use of all available neighboring
pixels especially for directional modes. For this, all the intra
predictors were updated to work with rectangular transform block sizes.

Performance improvements are small but free of cost:

All Intra frames:
lowres: -0.126
midres: -0.154

Video Overall:
lowres: -0.043
midres: -0.100

[Could not get AWCY results due to a backlog.]

BUG=aomedia:551

Change-Id: I7936e91b171d5c246cb0a4ea470a981a013892e6

This change does not impact the bitstream
  PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000

Change-Id: I6e131e91bad5efa345ed2542ae970eb6122eff51

Change-Id: Ideaeb52dbaf87e5a68da90cb94b0517760cb9d5c