Change-Id: I35b2b69e3937e70a7923ba76735f035f366de27f

Change-Id: I00554b925c0a870e766bc116c6d9cb02bd47a101

Change-Id: I785a0fbef18adf7cbdbd4f09117d132c7fe69156

Move most of the code of inverse_transform_block to
av1_inverse_transform_block such that encoder can use the
function as well.

Use av1_inverse_transform_block in av1_encoder_block_intra.
This will make the code cleaner.

Change-Id: I84dbeef2c65e252910606dbea446ce43165d504c

This will simplify the code flow in av1_subtract_txb()

Change-Id: I575da4a74f435b5862d2a848bdedae9856672121

A simpler interface for intra block prediction

Change-Id: I97e3f385746928d87999e6862b1d71a197df0302

This function help us merge subtract block code in
av1_subtract_plane and av1_encode_block_intra.

Change-Id: Ie793d88a218f1082c6fe28900a521f461e34d564

This reverts commit c538d681.

Reason for revert: should be av1_fwd_txfm(), instead of fwd_txfm()

Change-Id: I58657e54eb0a1c20c930b32cd53b6d05493eb8f4

Luc Trudeau authored Mar 10, 2017 and Yushin Cho committed Mar 29, 2017

The forward transforms required for PVQ are only performed if the block
is not skipped.

Change-Id: I22de3c7855ed800578a60aa06992cebea8bda119

Change-Id: Ie18fd2b8a3caf3948748ee353fe41e37f5803ba3

1) Add txb_entropy_ctx into MACROBLOCK_PLANE and PICK_MODE_CONTEXT

2) Add av1_get_txb_entropy_context() to compute the entropy context

3) Compute and sore the entropy context before av1_xform_quant()
return

Change-Id: Ia2170523af3163b9456f7c6a305c1e77ad2b23be

Entropy context of above and left block is not used
when PVQ is turned on.

Change-Id: I50c908bba800382438a48f7cc83f8b0f177393a1

Change-Id: Ida4a10fae54591f8f5b740739ab71a2650303958

Add av1_set_txb_context to perform context setting.
This will make the code cleaner and facilitate
lv_map experiment implementation

Change-Id: Iea7e1e8ff6820e2c9cf72961adaa58346a7eca75

Frankly I don't know what this was exactly doing in Daala
but it cannot be supported in AV1 because
state.sb_q_sacaling is not defined.

Change-Id: I75cb9080aa2a4af544b436b07ddcb9c27b3d25c4

Change-Id: I4d43d33725a5a0e6fdfa1168d1397cb122366b19

Change-Id: Id7cbb89ba344770297ef42bf1b09a2b9ba5a1378

Bring the following libvpx commits to aom:
e446ffd Cache optimizations in optimize_b()
50d3629 Repack vp9_token_state

Saves 24600 bytes of stack in the default configuration.

Change-Id: If9d6506cf3fe1c34ab639dedb3ef62a996293781

Fixes some rd-debug mismatches coding cat6 tokens with tx size < 32x32.
For these tokens the high extrabits are elided during tokenization and
detokenization, but the rd cost was computed with the old tables from
VP9 where these high extrabits are always coded.

Change-Id: I4a9a6ea822ff821e1932c351d43a57bdb4d6d466

Modify av1_coef_cost to reflect that EOB values are sent
with each non-zero coefficient, and modify chroma weight
to preserve original chroma balance.

AWCY, objective-1-fast, new_token+ec_adapt vs ec_multisymbol:

PSNR YCbCr:     -0.43%      -0.52%      -0.58%
PSNRHVS:        -0.81%
SSIM:           -0.73%
MSSSIM:         -0.93%
CIEDE2000:      -0.36%

This change:

PSNR YCbCr:     -0.15%      0.83%      0.77%
PSNRHVS:        -0.24%
SSIM:           -0.23%
MSSSIM:         -0.24%
CIEDE2000:       0.24%

Change-Id: I7f2dd9f3cd24bb64eb4131cba94dda466b9691b0

Cosmetic change that moves the declaration of mbmi at the beginning of
the function.

Change-Id: I028c9ca3db57e769270fbad4f20261d5562be2f2

Change-Id: Ie836a113978028f3bde2acd31061d9a663547087

Change-Id: Ifcdd3ce2953c1ecb1d0962da412a4b5ba2cda912

Since we now require C99, this is undefined behavior.
Thanks to Luc Trudeau for the report and Alex Converse for the
suggestion on how to make the macro safe for all integer sizes.

Change-Id: I99a1342dfedb3e17a6869269be317c2ed26bfe9b

Adds the static inline function get_plane_type to convert a plane number
to the corresponding PLANE_TYPE.

There's no change to the bitstream, it only encapsulates the logic to
get the PLANE_TYPE.

Change-Id: I1199db3a32c89437d9c029ab5b2b2e62582a13a2

Support the use of 2x2 prediction and transform block size for
chroma components. This experiment builds on top of cb4x4. It
brings back the loss in performance gains due to 4x4 chroma
component constrain. The coding gains for lowres are up from 1.5%
to 2.5%.

Change-Id: I8e461e230c2d920f6df1af9d935ebe9dfa3820d9

Properly scale the chroma component block size to handle the sub8x8
coding block in the encoding stage. It resolves the large performance
drop issue in cb4x4 due to minimum 4x4 block size.

Change-Id: Ia2477b43521408b72a20aca406f495faba66c033

Use the top-left 4x4 luma block's coding information for chroma
component encoding at size 4x4.

Change-Id: I4bcfbc2bf8b71f7fc30094553503468460a56f9b

Which was a workaround to balance the quality between
luma and chroma when PVQ is enabled, by
1) skipping PVQ DC RDO for chroma and
2) using smaller lambda value (x0.5) for intra prediction of chroma,
intended to spend more bits and thus improve quality for chroma.

The change in metric for 5 frames of objective-1-fast is:
PSNR  PSNR-HVS  SSIM  CIEDE-2000  PSNR Cb PSNR Cr  MS-SSIM
-1.45 -1.54     -1.46 4.62        13.71   14.87     -1.47

For still images (subset1)
PSNR  PSNR-HVS  SSIM  CIEDE-2000  PSNR Cb PSNR Cr  MS-SSIM VMAF
-1.52 -1.54     -1.60 2.56        11.92   12.17     -1.52  -1.66

Change-Id: I31c4d7a7de2939f6004c112649f7303f0e944156

Since token coding has changed the relationship
between EOB values and non-zero values, coeff costs
and trellis quant need to change to be more accurate.

Change-Id: I27ef400e8290db4c5faa9c21a253575eea7955c4

clang-format authored Feb 15, 2017 and James Zern committed Feb 15, 2017

Change-Id: Ib1b5dde5d4c6479c968cd7d7a4a1d01abae1fcf6

Change-Id: I1bf9002c159563d0e5238b26d1515ef305d416dc

Change-Id: Iad2b526d65865cbcb2119aca21686563ca8e97fd

This ensures we operate at the same precision that Daala uses, which matters
when activity masking is enabled, because of the gain companding.

Metrics from Patchset 4 (which had slightly incorrect rounding):

With activity masking (5 frames only):
av1_pvq_AM_ref_5f@2017-02-07T03:37:53.702Z -> av1_pvq_AM_derf_fix2_coeff_scaling_5f@2017-02-07T00:12:24.427Z

    PSNR |  PSNR Cb |  PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
  0.6653 | -12.3177 | -12.1858 |   0.3350 | 4.1013 |  2.0964 |    -4.0539

In particular for Netflix_Crosswalk_1920x1080_60fps_8bit_420_60f.y4m
 -5.0589 | -22.3077 | -21.2188 |  -7.0389 | -3.3715 |-5.7794 |   -13.1891

I.e., it fixes the large regression with AM on this sequence, and
 substantially improves chroma (at a lesser cost to other metrics).

Without activity masking (5 frames only):
av1_pvq_ref_5f@2017-02-07T03:52:51.279Z -> av1_pvq_derf_fix2_coeff_scaling_5f@2017-02-07T00:12:48.873Z

    PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
  0.0989 | -0.0322 | -0.0464 |   0.1883 | 0.0795 |  0.0579 |     0.0923

Change-Id: I46b808b7c8e4733465f8bebc8336dfd5b75783ec

Creates the PVQ_SKIP_TYPE enum to encapsulate the different types of
skipping that can be signaled by PVQ (i.e. skip: AC, DC or both).

There is no impact on the bitstream. However, the decoder will now emit
an internal error if the decoded skip flag is out of range. The
block_skip variable is also renamed to ac_dc_coded as it stores the same
information.

Change-Id: Ib2aadaf99dc1736ea392ae5ed8948c3cdc12da9b

Now that https://aomedia-review.googlesource.com/#/c/6729/
has been merged, build_intra_predictors_for_interintra() is
now redundant, so replace it by a direct call to
av1_predict_intra_block() and remove the old function.

Reset rect_interintra back to 1.

To do this, we need to make the intra predictor take a
BLOCK_SIZE instead of a TX_SIZE. This is because we need to
be able to predict 32x64 and 64x32 blocks, but there is no
TX_32X64 or TX_64X32.

No effect on output or performance.

Change-Id: I8c185a211c97a85012cc54ec293c785a693608ed

This commit adds a new experiment, Daala's distortion function,
which is designed to better approximate perceptual distortion
in 8x8 pixel blocks.

This experiment is expected to work best with PVQ.

It measures the variance of overlapped 4x4 regions in the 8x8 area,
then uses these variances to scale the MSE of weighted frequency domain
distortion of 8x8 block.

Since AV1 calculates distortion in blocks as small as 4x4, it is not possible to
directly replace the existing distortion functions of AV1,
such as dist_block() and block_rd_txf().
Hence, there has been substantial changes in order to apply
Daala's 8x8 distortion function.
The daala distortion function is applied
after all 4x4 tx blocks in a 8x8 block are encoded (during RDO),
as in below two cases:
1) intra/inter sub8x8 predictions and
2) 4x4 transform with prediction size >= 8.

To enable this experiment, add '--enable-daala-dist' with configure.

TODO: Significant tuning of parameters is required since the function has
originally came from Daala thus most parameters would not work
correctly outside Daala.
The fact that chroma distortion is added to the distortion of AV1's RDO is
also critical since Daala's distortion function is applied to luma only
and chroma continues to use MSE.

Change-Id: If35fdd3aec7efe401f351ba1c99891ad57a3d957

Instead of returning skip, av1_pvq_encode_helper and od_pvq_encode now
return ac_dc_coded. This gives more information on whether the DC part
or the AC part was skipped.

Although it is possible to obtain ac_dc_coded from the pvq_info struct,
this struct is not always used, in which case the information was lost.

This change does not impact the bitstream.

Change-Id: Ie303de915f74e8da384f822332eb1aa27f677bd3

These are under EXT_TX + RECT_TX experiment combo.

Results
=======

Derf Set:
--------
All Intra frames: 1.8% avg improvement (and 1.78% BD-rate improvement)
Video: 0.230% avg improvement (and 0.262% BD-rate improvement)

Objective-1-fast set
--------------------
Video: 0.52 PSNR improvement

Change-Id: I1893465929858e38419f327752dc61c19b96b997

This commit reworks the transform block partition context update
to support cb4x4 mode in the recursive transform block partition.
It resolves the remaining enc/dec mismatch issue when both cb4x4
and var-tx are turned on.

Change-Id: I850d121204fe4c68e81488f1d2848c570d9d08b9