GitLab

Generalize quantizer setup so that quantization and TX can use
different coefficient shifts/scalings without inpacting RDO lambda
generation.

This patch is documentaiton + a minor refactor setting up later
work; it causes no functional change.

monty-daalaTX-fulltest-Daalabaseline-o1f@2017-11-07T00:01:20.779Z ->
 monty-daalaTX-fulltest-DaalaRDO-o1f@2017-11-07T00:02:31.347Z

PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | CIEDE 2000
 N/A |  0.0000 |  0.0000 |      N/A |  N/A |     N/A |        N/A
(note-- the numbers above were collected using --cpu-used=3, which
 appears to be newly broken.)

monty-daalaTX-fulltest-Daalabaseline-o1f4@2017-11-07T05:59:16.553Z ->
 monty-daalaTX-fulltest-DaalaRDO-o1f4@2017-11-07T05:59:50.180Z

  PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000

monty-daalaTX-fulltest-Daalabaseline-s1@2017-11-07T00:01:46.582Z ->
 monty-daalaTX-fulltest-DaalaRDO-s1@2017-11-07T00:02:56.282Z

  PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000

Change-Id: Ia5a3c052eacd17184ca1b0fe6d032cfb9afdb77f

To skip possible unnecessary looping.

Change-Id: If611d344ab39eee53de6c5256ce5f8febedecc9b

Change-Id: I8b3e028b3c17660b390ce9c65b3d31aabc83efd9

av1_txb_init_levels() has prefix 'av1' because it will be optimized
and not static later.

Change-Id: I988da817335f122522a76c4412207c3a6fdd6b71

Removed assertions were not properly set up and may cause decoding failure when USE_CASUAL_CTX is enabled. This CL does not change bitstream.

Change-Id: Ib9193cbda32f342335a79aca39e9cc49204a0ec9

This CL simplifies context derivation for nz and base level flags in
level map.
1. Reduce SIG_COEF_CONTEXTS from 58 to 42.
2. NZ and base level flags share the same context offsets derived from a
template of size 5 (down from 7).

In limited runs, compression performance seems neutral if not better.
Encoding time for a key frame on a local linux machine is reduced by about 25% or more.

Change-Id: Ibd93b21c839154bc5ae26b993f9e66537cbf5942

Change-Id: I12471d18544a9634c28ed139b7211c9af0762927

Account for 1-D/2-D transform kernels for the eob modeling. To
maintain a smaller context cardinality, set the two 1-D transform
kernels in the same category. The difference in directions should
be largely covered by the scan order.

This and the previous CLs on nz_map context modeling together
improve the compression performance of level-map coefficient coding
system by 0.4% for lowres.

Change-Id: I8c4f03ca01ce3d248950d04bd1266f445b4227a0

This commits moves coding of signs in a transform block into its own
pass. Grouping signs coded in literal mode together may help improve
throughput.

Change-Id: I999c6eeeb780b188ab9e2c6ea888f9de4dbce893

Change-Id: Ib36a8df1a3ebddbf4320fb7b9b5537041bddc3a3

Use br-node approach, which can be easily turned into multi-symbol
if desired.

Change-Id: I40df5178ab299af24d347d91f01a88dbfc9305a6

Change-Id: I2ae2a33574bc3072561e696a31e0ea2e0770afa9

BUG=aomedia:954
BUG=aomedia:956

Change-Id: Idc292c524bc3fea2a8a0e003d1a4cc9ae5799bd1

Change-Id: I32c5015bedef07f58413c679b4aaf30651a357b9

Change-Id: Ib0e0b5f7085d3c910dcd15a3628a001af864c8fc

This will avoid overflow issue when coeff uses more than 8 bits

BUG=aomedia:954


Change-Id: Ib5f323b0d9db3141cc019a6aad019f09b8c4f9fa

Change-Id: Ia67beabb85eedd5da5eb69f434dde71f66f08006

This commit makes the following changes to level-map coding
	1. Encode and decode eob position first
	2. Context derivation for non-zero flags
	3. Single pass in optimize-txb

lowres 0.33% gain

Change-Id: Ia2b6cfd70425f49ac021454d42d5950bfcfab50a

Let read_nz_map() etc. output to uint8_t*.

Temporarily introduced coeff_is_byte_flag as an argument of
get_nz_count() and get_nz_map_ctx() to handle different types of input
coefficients. It helps to not duplicate too many functions.
If possible, unify types to uint8_t* later.

Change-Id: Idbbe67b7ec563f8c9299daa0fa7d05e15c8295e6

Since CB4X4 is adopted and without it the codec does not work,
it is better to remove it and simplify the code.

Change-Id: I51019312846928069727967e3b2bbb60f0fba80d

Remove the unused argument.

Change-Id: I4b7936e6c12a9d0a54c9d1cc281f96ec623e5cd8

Split coefficients into signs (0 or 1) and levels (0 to 255),
so that they both can be fit in 1-byte.

Change-Id: I0f486368b7b819a77aaddda4710e83189e53fc55

This aims at facilitate the experiment about reduce context
dependency

Change-Id: I3d026bda1118cf613001efa32deed62997d5e3bb

Remove exit() from the source code.

Change-Id: Id49cf65e2131ecf5297e147c4954e448dd42d805

bug=aomedia:837

Change-Id: I54233963eac76d7667d5b1350f49ef3108027708

Experiment probability precision for binary coding in the lv-map
coding system.

Change-Id: I8d9c49eee6dc7ca7970390fa5febe25b80bfab3c

Change-Id: I751a6719e08d18ea45ee902b783bf3b3c830cc58

Change-Id: Ibdf1de139e8ce63b0be2d0cd1fd098f06739d8cc

Change-Id: I03adc36f5e5f839d23c4f50a864d317d008e30a6

Observe 0.1% gain on lowres wo optimize_b before rebase

Change-Id: I0cb5b5e4be2563093efb2f6dfbefdce9b554e910

Change-Id: I3209d0074bf3ed1fda260dc2f5cacbfa616ad6ae

Change-Id: Ie52d52bc25e3fdfdea877349215431d8edc064a3

Fix the index order of br_idx.

Change-Id: I55acda816d1ec33c25ca4c31b09976f5e6f9c562

Replace the truncated geometric distribution model with the grouped
leaves structure for more efficient probability modeling.
Each group has its own Geometric distribution

This give us 0.2% gain on lowres

Change-Id: If5c73dd429bd5183a8aa81042f8f56937b1d8a6a

BUG=aomedia:719

Change-Id: Id47c2c68cb61e142ce62fee5e97f7c474b3dc273

This will let coding performance drop slightly
lowres 0.093%

Increase encoder speed by 24%

Reduce nz_map's context size by 20%

Change-Id: I871c18a7e0341e066afc334556b9998194b3f8c9

Change-Id: I7b0e998182b522274768e4b587080d8e88f3a223

Support the per symbol cdf update at the level map syntax element
decoding process.

Change-Id: Id58bf86fb1c7c72dd14ddaa3781caaa27f446ef7

Change-Id: Ibd0bda593717aab2894c36dd59925a0bcf654113

Use the row and column indexes to fetch txk_type, which allows the
chroma components to derive the tx type from the corresponding luma
components. It improves the coding performance of txk-sel by 0.18%.

Change-Id: I3f4bca5839e13ae95e51053e76cd86fe58202ac9