GitLab

It stopped being used after 09302f5a

Change-Id: Ie7d567c787a4120f8b73378b3a82267249a82e3d

The function cfl_compute_parameters is added and contains the logic
related to building the CfL context parameters. As such, many cfl
functions can now be encapsulated inside of cfl.c and not exposed to the
rest of AV1.

This also allows for supplemental asserts that validate that the CfL
context is properly built.

Results on Subset1 (compared to 9c6f8547 with CfL)

  PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000

Change-Id: I6d14a426416b3af5491bdc145db7281b5e988cae

Change-Id: Iaae46d0735539b8b8daf9faac81c2a3434838020

Change-Id: Ie721eaf58d0716e340b9ebdff9fd215cfe0c3c2a

This change does not impact the bitstream, it changes how to distortion
is summed when evaluating alpha. The sum is still taken over the entire
partition. However, instead of iterating over the entire surface all at
once, CfL now iterates over each transform block. This is in light of
future work to compute alpha over transform blocks and not prediction
blocks.

Results on Subset1 (compared to 9c6f8547 with CfL)

  PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000

Change-Id: Ic7b72201d29ad6b2527748e35b212bec515e3bdb

This is to facilitate future experiment related to adapt_scan

Change-Id: I51628f3df81bd82db7f8f553d13da0ee5792d7d9

cb4x4 itself should not require these sizes.

This simplifies compatibility with other experiments, since we can
first make them work with cb4x4 (which is now on by default), and
then worry about chroma_sub8x8 and chroma_2x2 (which is not) in
separate steps.

Encoder and decoder output should remain unchanged.

Change-Id: Iff2a5494cab3b7d96f881e8bd9cd4bf18c817cfa

Results on Subset 1 (Compared to a0f8c145 with CfL)

  PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
0.0677 | -0.3359 | -0.2115 |   0.0529 | 0.0735 |  0.0495 |    -0.0907

Change-Id: Ib61ff862e8cfbdf0c693a4eba5f2712a6e9ab819
Signed-off-by: David Michael Barr <b@rr-dav.id.au>

Extract the compution of the luma reconstructed average out of cfl_load
and into cfl_compute_average. The reconstructed luma average is stored
in the CFL_CONTEXT to avoid computing it for each transform block and
for each plane.

Results on subset1 (compared to 803bea26 with CfL)
   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.0474 | -0.1486 | -0.2931 |  -0.0358 | -0.0397 | -0.0127 |    -0.1162

Change-Id: I9e34af0fe5961ce8dbe70cb80aea2a16221d0d92

This was initialized from a const and never modified. But was still
passed around and stored in multiple structs.

Removed these 'rddiv' member variables and now RDOPT() and RDOPT_DBL()
always use the const RDDIV_BITS directly.

Change-Id: I1a8dfd2c8fa857d466ad1207b4f0dd6ec07eafb8

Before:
-------
accu_error was always <= 0:
- It was initialized to zero, and
- Became more and more negative through the iterations.

This meant that RDOPT() macro usage on accu_error was causing left shift
of negative value.

Now:
----
- accu_error is initialized to a large positive value.
- It becomes smaller on every iteration but remains >= always.

This is functionally equivalent: verified that there's no change in
output due to this patch.

BUG=aomedia:600

Change-Id: I654572a72234c588c8f332d5b66a5587fc6610b4

Make the level-map and var-tx work together.

Change-Id: I5ff80a3cbc43af64b504934209720f387b89dbd8

Explicilty use the transform block size to determine the coeff
band array. Remove the assumption on square transform block size.

Change-Id: I18e285130465a5eced49304a27a6cb617e297760

Map the rectangular transform block size into the bigger square
transform block size as the context for level map probability
model.

Change-Id: I20cf2b16daec16172855a78a201b670ff0547bf5

plane_bsize is now computed properly. This also includes support for the
special case of blocks < 4X4

Results on subset1 (compared to 8e689e4b with CfL)
   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.0218 | -0.2328 | -0.2555 |  -0.0230 | -0.0379 | -0.0723 |    -0.1205

Change-Id: I6ec87d818d8df6a40ecf3bb1b86954e59c952930

In function av1_xform_quant().

For subset1, high delay mode:
   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.0565 | -0.0975 |  0.0317 |  -0.0937 | -0.0930 | -0.1261 |    -0.0994

Change-Id: I638c2a4c2ad359709069c1587b677a813fb589c2

Only apply quantisation matrices when the transform is 2D.

Change-Id: Iae9ac910c11199f7944a80d10d334db3b711059d

This avoids the rounding errors due to the right shift of the
negative numbers that cause the reconstruction coefficient has
higher distortion than the source coefficient.

BUG=aomedia:599

Change-Id: I11ed86bf1d41164dda4398545334a7b4e8e10513

This has been found to be better than the original version in both ways:
(1) Better compression: lowres -0.229, midres -0.147
(2) Faster too in my quick test over 5 different clips with 30
frames: 2.7% to 10.5% faster.

Change-Id: I4d46e0915d6e4b8e7bfc03d0c8b88cbe3351ca20

This unifies the codepath for high-bitdepth transforms and deletes
all calls to the old deprecated versions. This required reworking
the way 1d configurations are combined in order to support rectangular
transforms.

There is one remaining codepath that calls the deprecated 4x4 hbd
transform from encoder/encodemb.c. I need to take a closer look
at what is happening there and will leave that for a followup
since this change has already gotten so large.

lowres 10 bit: -0.035%
lowres 12 bit: 0.021%

BUG=aomedia:524

Change-Id: I34cdeaed2461ed7942364147cef10d7d21e3779c

Comparing to lv_map experiment using the old av1_optimize_b
function, This change gives 0.8% gain on lowres and midres
datasets

Comparing to the baseline with greedy optimize_b, the lv_map
experiment will have 1.2% on lowres and 1.49% on midres.
The gains are similar to the one that under the condition of
turning off av1_optimize_b for baseline and lv_map.

Change-Id: Ie17c4ecbd9ed8418ef657a526d8363e0746b5af5

This is to facilitate lv_map experiment

Change-Id: Ife779b172c4b81a9b2b4640464163300996e3969

This patch forces DAALA_EC on by default and removes the dkbool coder.

Change-Id: Icd2ff08efd7bf467adf554344111473cb357adf8

In both versions of av1_optimize_b, the code to calculate the
dequantized coefficient from a modified quantized coefficient
was not compatible with tx64x64.

Specifically, the new dequantized coefficient was calculated as an
offset applied to the original coefficient. This had a fixup block
for 32x32 transforms, but not for 64x64 transforms.

For simplicity, instead of extending the fixup block, we now
recalculate the dequantized coefficient directly.

BUG=aomedia:537

Change-Id: I86c1c8820f324ad3f6afd6a720ff59f754384a19

Change-Id: Ibbf0c14934d0bff316cfdf5c252c64b3e68c73e7

Separate the codes into a table of distinct values
and an index into that table.
Pull the SSE calculation of the RDO loop and avoid
repeating for the same alpha values.

Change-Id: I8c4bd7eab6f8000e6aca9687d9190abc3e270c37
Signed-off-by: David Michael Barr <b@rr-dav.id.au>

Encode a block line by line, horizontally or vertically. In the vertical
mode, each row is predicted by the reconsturcted row above;
in the horizontal mode, each column is predicted by the reconstructed
column to the left.

The DPCM modes are enabled automatically for blocks with horizontal or
vertical prediction mode, and 1D transform types (ext-tx).

Change-Id: I133ab6b537fa24a6e314ee1ef1d2fe9bd9d56c13

Change-Id: Ia0eb3a3694fdbe9d33548ff4014b704b2f3db86a

Change-Id: I443004a9bdc026620f364f5bc1dcc1d5e0e4ca6f

Encapsulates the logic to update the rate of each CfL codeword.
The if statements are removed from the loop and the arrays are
stored in CFL_CTX instead of being declared every time.

Change-Id: I0cb208b14e6c6a888210dd33c5e8fe8d74dd87f4

Change-Id: I0ed6d8f2b6a83641f65096cf41fcde94e70d94d4

Change-Id: I3e3d458aec7e26abc366992ac20f9fa838fe35bb

Change-Id: Id14d83bdce8321bef81972a4f5530a50589d9578

EC_ADAPT stores the count at the last position. As such, the CDF_SIZE
macro adds 1 to the array size. This resolves valgrind warnings about
jumps on uninitialized variables.

Change-Id: I58d607a57756cb166a9ae5c4565537ce202183fd

CFL is disabled when skip_chroma_rd is enabled. This is done by reusing
the logic in CB4X4. To facilitate integration, the skip logic used in CfL is
moved inside the read/write functions.

Results on Subset1:
master@2017-05-08T19:54:48.196Z -> cfl_baseline@2017-05-08T20:06:55.292Z
  PSNR |  PSNR Cb |  PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
0.2668 | -12.0951 | -10.4138 |   0.3095 | 0.2998 |  0.2831 |    -3.6579
https://arewecompressedyet.com/?job=master%402017-05-08T19%3A54%3A48.196Z&job=cfl_baseline%402017-05-08T20%3A06%3A55.292Z

Change-Id: I45644baa1aceef5ad4da3332fcb10f3fbaac052b

Since the size used with cfl_load can either be based on the transform
block size and the prediction block size, width and height are used as
parameters instead of TX_SIZE.

This resolves a problem where cfl_compute_alpha_ind was reading
uninitialized memory.

Change-Id: I187dbdd5b2e8bd85e82bb77eb74859bee2cd3f1e

This will improve the lowres performance by 0.1%
No significant change on midres and hdres

Change-Id: I0c6e69bf48596e7e5ad49631ed9f9be73bbf3c9a

Measure SSE for all possible alphas.
Estimate rates for alpha signalling.

Change-Id: Idf1e3c632925cd306090fc38cf5b95eff7ee5c1c
Signed-off-by: David Michael Barr <b@rr-dav.id.au>

The conversion of return value to int64_t appears to be unnecessary.

fixes a number of type conversion warnings for MSVC2015

Change-Id: Ibca24468e73bf22ad07c469ce5d1223635390ecb

Removed "const" from parameters that are passed by value, also fixes
msvc2015 compiler warnings.

Change-Id: I0c7555b4245fa55fabda1c27762fb9f0860bfa18