This avoids the rounding errors due to the right shift of the
negative numbers that cause the reconstruction coefficient has
higher distortion than the source coefficient.

BUG=aomedia:599

Change-Id: I11ed86bf1d41164dda4398545334a7b4e8e10513

This has been found to be better than the original version in both ways:
(1) Better compression: lowres -0.229, midres -0.147
(2) Faster too in my quick test over 5 different clips with 30
frames: 2.7% to 10.5% faster.

Change-Id: I4d46e0915d6e4b8e7bfc03d0c8b88cbe3351ca20

This unifies the codepath for high-bitdepth transforms and deletes
all calls to the old deprecated versions. This required reworking
the way 1d configurations are combined in order to support rectangular
transforms.

There is one remaining codepath that calls the deprecated 4x4 hbd
transform from encoder/encodemb.c. I need to take a closer look
at what is happening there and will leave that for a followup
since this change has already gotten so large.

lowres 10 bit: -0.035%
lowres 12 bit: 0.021%

BUG=aomedia:524

Change-Id: I34cdeaed2461ed7942364147cef10d7d21e3779c

Comparing to lv_map experiment using the old av1_optimize_b
function, This change gives 0.8% gain on lowres and midres
datasets

Comparing to the baseline with greedy optimize_b, the lv_map
experiment will have 1.2% on lowres and 1.49% on midres.
The gains are similar to the one that under the condition of
turning off av1_optimize_b for baseline and lv_map.

Change-Id: Ie17c4ecbd9ed8418ef657a526d8363e0746b5af5

This is to facilitate lv_map experiment

Change-Id: Ife779b172c4b81a9b2b4640464163300996e3969

This patch forces DAALA_EC on by default and removes the dkbool coder.

Change-Id: Icd2ff08efd7bf467adf554344111473cb357adf8

In both versions of av1_optimize_b, the code to calculate the
dequantized coefficient from a modified quantized coefficient
was not compatible with tx64x64.

Specifically, the new dequantized coefficient was calculated as an
offset applied to the original coefficient. This had a fixup block
for 32x32 transforms, but not for 64x64 transforms.

For simplicity, instead of extending the fixup block, we now
recalculate the dequantized coefficient directly.

BUG=aomedia:537

Change-Id: I86c1c8820f324ad3f6afd6a720ff59f754384a19

Change-Id: Ibbf0c14934d0bff316cfdf5c252c64b3e68c73e7

Separate the codes into a table of distinct values
and an index into that table.
Pull the SSE calculation of the RDO loop and avoid
repeating for the same alpha values.

Change-Id: I8c4bd7eab6f8000e6aca9687d9190abc3e270c37
Signed-off-by: David Michael Barr <b@rr-dav.id.au>

Encode a block line by line, horizontally or vertically. In the vertical
mode, each row is predicted by the reconsturcted row above;
in the horizontal mode, each column is predicted by the reconstructed
column to the left.

The DPCM modes are enabled automatically for blocks with horizontal or
vertical prediction mode, and 1D transform types (ext-tx).

Change-Id: I133ab6b537fa24a6e314ee1ef1d2fe9bd9d56c13

Change-Id: Ia0eb3a3694fdbe9d33548ff4014b704b2f3db86a

Change-Id: I443004a9bdc026620f364f5bc1dcc1d5e0e4ca6f

Encapsulates the logic to update the rate of each CfL codeword.
The if statements are removed from the loop and the arrays are
stored in CFL_CTX instead of being declared every time.

Change-Id: I0cb208b14e6c6a888210dd33c5e8fe8d74dd87f4

Change-Id: I0ed6d8f2b6a83641f65096cf41fcde94e70d94d4

Change-Id: I3e3d458aec7e26abc366992ac20f9fa838fe35bb

Change-Id: Id14d83bdce8321bef81972a4f5530a50589d9578

EC_ADAPT stores the count at the last position. As such, the CDF_SIZE
macro adds 1 to the array size. This resolves valgrind warnings about
jumps on uninitialized variables.

Change-Id: I58d607a57756cb166a9ae5c4565537ce202183fd

CFL is disabled when skip_chroma_rd is enabled. This is done by reusing
the logic in CB4X4. To facilitate integration, the skip logic used in CfL is
moved inside the read/write functions.

Results on Subset1:
master@2017-05-08T19:54:48.196Z -> cfl_baseline@2017-05-08T20:06:55.292Z
  PSNR |  PSNR Cb |  PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
0.2668 | -12.0951 | -10.4138 |   0.3095 | 0.2998 |  0.2831 |    -3.6579
https://arewecompressedyet.com/?job=master%402017-05-08T19%3A54%3A48.196Z&job=cfl_baseline%402017-05-08T20%3A06%3A55.292Z

Change-Id: I45644baa1aceef5ad4da3332fcb10f3fbaac052b

Since the size used with cfl_load can either be based on the transform
block size and the prediction block size, width and height are used as
parameters instead of TX_SIZE.

This resolves a problem where cfl_compute_alpha_ind was reading
uninitialized memory.

Change-Id: I187dbdd5b2e8bd85e82bb77eb74859bee2cd3f1e

This will improve the lowres performance by 0.1%
No significant change on midres and hdres

Change-Id: I0c6e69bf48596e7e5ad49631ed9f9be73bbf3c9a

Measure SSE for all possible alphas.
Estimate rates for alpha signalling.

Change-Id: Idf1e3c632925cd306090fc38cf5b95eff7ee5c1c
Signed-off-by: David Michael Barr <b@rr-dav.id.au>

The conversion of return value to int64_t appears to be unnecessary.

fixes a number of type conversion warnings for MSVC2015

Change-Id: Ibca24468e73bf22ad07c469ce5d1223635390ecb

Removed "const" from parameters that are passed by value, also fixes
msvc2015 compiler warnings.

Change-Id: I0c7555b4245fa55fabda1c27762fb9f0860bfa18

Writes and reads alpha to and from the bitstream.

A special case is needed on the encoder side to handle prediction block
skips. Since whether or not a prediction block is skipped during CfL, a
rollback is required if the block was skipped and the alpha index was
not zero. The advantage of this is that no signaling is required when
the prediction block is skipped as it is assumed tha the alpha index is
zero.

A encode facade is added to the intra prediction facade as CfL requires
special encoder side operations.

Change-Id: Ic3b11d0fdbd51389d862112eb09d8785127a6b06

This will guarantee that av1_optimize_b will be turned off when
lossless mode is on

Remove heuristic lossless check in optimize_b_greedy

Change-Id: I636c776f3f6b632eb03bc57a470ea43aae4fe0f6

This also avoids having 'bit_depth'/'bd' members whose presence
is decided by the preprocessor.

The goal is to simplify statements accessing the bitdepth value.
(instead of forcing them to also rely on the preprocessor to query
the member presence through #if CONFIG_HIGHBIDEPTH).

Change-Id: I68b822bbaa2b1c67bde81cb4f3f8b9988b798b0d

BUG=aomedia:443

Change-Id: Iebdb609564c2c3040f3902052afeeec7ae83412d

Stores the reconstructed luma pixels for each transform block inside a
prediction block. Rectangular transform blocks are supported.

As for RDO, after all the modes have been tested for luma, an extra
encoding is perform in order to store the reconstructed pixel values of
the best mode. These values are then used for RDO on the chromatic
planes.

Change-Id: I354d9827e32fd41065f1b2ce02832d943a6fa156

inline is undefined in visual studio 2013 for C

Change-Id: I85adb3968e4a98e2d7909cc42e955b1447fcfa26

This is the first patch for ext_delta_q experiment.
* Allow delta q to work with segment features

Change-Id: I9455ebb3b5b23b41daa4afa1149672a20d3a3f2c

This fixes crashes due to infinite recursion when var_tx,
ext_tx and rect_tx are enabled without cb4x4. This
is the first part of an ongoing fix for this experiment
interaction.

Change-Id: I674f28294666102aff2265f6b6112816cac17378

The greedy search method improves the BD-rate over the baseline by
more than 0.2% for lowres test set. Larger gain 0.55% is observed for hdres test set.

[2017.04.06] Cleanup to remove redundant computation. On a local linux
machine, the greedy method is now faster than the trellis method in
encoding the first 100 frames of foreman_cif. However, the BD-rate seems
to become smaller due to the recent changes to the codebase.
[2017.04.06-2] Style changes to meet the requirements.
remove "greedy-optimize-b" in configure
[2017.04.10] Move the changes under the macro USE_GREEDY_OPTIMIZE_B
[2017.04.11] Adjust rdmult to accommodate CpuSpeedTest
[2017.04.12] Set USE_GREEDY_OPTIMIZE_B to 0 at the request of debargha@.
[2017.04.13] Move greedy implementation of optimize_b into a separate
function with the same name (selected by USE_GREEDY_OPTIMIZE_B, default
is 0)

Change-Id: Ic15534f8d696d5f02e8f2e3e9f08b52e41e9efd2

Sebastien Alaiwan authored Apr 12, 2017 and James Zern committed Apr 12, 2017

Rename '--enable-aom-highbitdepth' to '--enable-highbitdepth'

Change-Id: I1de13c3508c30c552532993419d8ace326142ab6

This was broken by commit 1238137c.

When calling the generic coder with a max, it truncates the CDF, but
then tries to encode/decode with the Q15 entropy coder functions
with built-in adaptation. That causes assertion failures, because the
total probability isn't 32768.

We could fix it by re-scaling the CDF, and then doing adaptation
assuming there was no max, but that requires several special-case
code paths.

Instead, since non-robust streams were the only thing that still
required calling the generic coder with a max, and since the gain
from them is very small (and they require doing more DSP just to be
able to parse the stream), we simply remove the option and force
the use of robust streams all the time.

With robust streams enabled, encoder output should not change, and
all streams should remain decodable without decoder changes.

Thanks to Nathan Egge for reporting the problem.

Change-Id: I6c81481abb796688bf703d68f164d208e6a69f20

Properly scale the chroma component size per color plane.

Change-Id: Ibf192d0e99f2fc3611beb82a3a9951ad09b292d3

1) block_raster_idx is actually raster order only when tx_size
is TX_4x8.
It's very specific, so we should put it near to the place it's
actually used.

2) Sync the meaning of block_idx on encoder/decoder sides

Change-Id: I7d37a992cb773503e29f9c0d9d2586e580aa6173

Make is_chroma_reference() account for all the YUV420, 444, and
422 formats.

Change-Id: Ia87e51894493dcea86843194a34e5de05799248a

Change-Id: I7eaff4fd5ee24d6e1138ad8d241dbb63a697961f

In the beginning of encoding and bitstream packing,
the frame context in common of codec is copied to the frame context
in each tile.
Initial prob and context is based on flat probability and
does not come from table at the moment.

The bd-rate change for the test set objective-1-fast
on awcy with high delay mode is:

PSNR   PSNR  HVS  SSIM  CIEDE 2000  PSNR Cb PSNR Cr MS SSIM
-0.77  -1.05      -0.74 -0.67       -0.67   -0.77   -0.88

Change-Id: Ic9105ac68aceb7486cb5f6f1c0b19df5853f2cb9

Change-Id: I35b2b69e3937e70a7923ba76735f035f366de27f