GitLab

Cleanup related unit-tests.

Change-Id: Ic756e6bbad80f5b9947ca1cdd55cdef77b985f81

We are going to have several commits to setup new low/high
bitdepth data path selection logic. This patch is for inverse
transform. Let me summarize the ideas as following.

- For low/high bitdepth selection, encoder depends on
  input configuration, e.g., video sequence bitdepth,
  profile. Decoder depends on input bitstream. This has
  nothing to do with compiler/build  configuration.

- Typical encoder usage for sampling format 4:2:0.
  1) 8-bit video sequence:
   a) --profile=0
   Fastest encoding/decoding pipeline on speedup.

   b) --profile=2 --bit-depth=10
   Image pixels are left shifted by 2 bits. It
   employs 16-bit reference frame buffer and has high
   calculation precision. It usually enjoys higher
   compression performance.

  2) 10/12-bit video sequence (HDR):
   --profile=2 --bit-depth=10/12

- Transform coefficient type:
  Lowbitdepth:  int16_t
  Highbitdepth: int32_t

- The type, tran_low_t is still used in codebase,
  Which is int32_t, defining the data path capacity.
  Naturally, it is high bitdepth.

Eventually we shall remove the configuration flags,
CONFIG_HIGHBITDEPTH/CONFIG_LOWBITDEPTH, and seperate
low and high bitdepth data path. Two data paths co-exist
in the same build environment.

Change-Id: I35c06d4d4f19ebf80d909168fdddbae57c3cc884

In previous ADSTs, DST-7 and DST-4 are used for length 4 and length
8/16/32, respectively. In this LGT experiment we explore transforms
between DST-4 and DST-7. When CONFIG_LGT flag is on, adst4 and adst8
are replaced by lgt4 and lgt8, the intermediate transforms with
pre-chosen parameters.

The LGTs applied here are lgt4_160 and lgt8_170, where the numbers
mean the self-loop weights times 100. The associated values for DST-7
and DST-4 are 100 and 200.

ovr_psnr:
lowres: -0.140
midres: -0.131
hdres: -0.078

These changes are not applied to the highbd scenario in the
current version.

Change-Id: I20600456da8766528b2b6b11aa28801e70af498e

Responding to some left over cosmetic comments from
2b5cdb1cf87c933331a16cc0221455d0a8c255e1

Change-Id: I42e126593526cedd6675adf35b9c1df78e1ddf54

This unifies the codepath for high-bitdepth transforms and deletes
all calls to the old deprecated versions. This required reworking
the way 1d configurations are combined in order to support rectangular
transforms.

There is one remaining codepath that calls the deprecated 4x4 hbd
transform from encoder/encodemb.c. I need to take a closer look
at what is happening there and will leave that for a followup
since this change has already gotten so large.

lowres 10 bit: -0.035%
lowres 12 bit: 0.021%

BUG=aomedia:524

Change-Id: I34cdeaed2461ed7942364147cef10d7d21e3779c

cb4x4 itself should not require these sizes.

This simplifies compatibility with other experiments, since we can
first make them work with cb4x4 (which is now on by default), and
then worry about chroma_2x2 (which is not) in separate steps.

Encoder and decoder output should remain unchanged.

Change-Id: I4e9fcdae49f238b5099a3c74a398fe993c2545f8

Encode a block line by line, horizontally or vertically. In the vertical
mode, each row is predicted by the reconsturcted row above;
in the horizontal mode, each column is predicted by the reconstructed
column to the left.

The DPCM modes are enabled automatically for blocks with horizontal or
vertical prediction mode, and 1D transform types (ext-tx).

Change-Id: I133ab6b537fa24a6e314ee1ef1d2fe9bd9d56c13

Exposed by Change-Id: I048c6e9cc790520247cc21ae9b92a9c8d84d00a7

BUG=aomedia:525

Change-Id: Ia83f8a8efcf0eac4912f247f38887c0dd533da85

This adds the proper cfgs to av1_{inv/fwd}_txfm1d_cfg for
the identity transform so all hbd transforms can use
the same codepath. This has no impact on performance
since the new identity transforms that correspond with
the cfgs are not yet being called. Once this is checked in,
we should be able to delete all deprecated transform functions
and have a single code flow for all hbd transforms.

BUG=aomedia:524

Change-Id: I3d1bfbc8bc29b367e8ddf7dcd27525af0bd31067

It gives 0.1% gain on lowres and midres

Change-Id: I555a492a68571c525713840d73aa5614fe80a87d

The hbd transform configurations were originally written for all possible
2d transforms. Now that there are many more possible 2d transforms
due to EXT_TX and RECT_TX, it is simpler to write the cfg for the
4 1D transform types and compose them to make all new possible transform
types. This will allow for an easier integration of the identity transform
for EXT_TX and rectangular transforms for RECT_TX into the current
hbd transform codepath and facilitate the removal of obsolete transforms.
This has no impact on performance.

BUG=aomedia:524

Change-Id: I1e217bcd217fd637b1df94fae62d9c59a0523c1a

- Add eob delivery in av1_first_pass().
- Fix eob threshold pointer copy.
- Change local functions in IDCT to static.

Change-Id: I1c95e8f899088773f063fe6be7f26a5f96864644

Change-Id: I3b30030b1b5f94b43b6dac7b2eac1dc1ea308e84

- Function level improvement (ms):
Functions       ssse3  avx2   Percentage
idct32x32_1024  794    374    52.9%
idct32x32_135   354    169    52.2%
idct32x32_34    197    142    27.9%
idct32x32_1     n/a     26    n/a

- Integrating in default scan order.

Change-Id: I84815112b26b8a8cb800281a1cfb1706342af57d

- Function level improvement:
functions      sse2  avx2  percentage
idct16x16_256  365   226   38%
idct16x16_38   n/a   136   n/a
idct16x16_10   171   110   35%
idct16x16_1     34    26   23%

- Integrated in AV1 for default scan order.

Change-Id: Ieb1a8e730bea9c371ebc0e5f4a748640d8f5e921

This reverts commit f9d77bd5.

Reason for revert: 8x8 transform failures

BUG=https://bugs.chromium.org/p/aomedia/issues/detail?id=502

Change-Id: I2f6c10bc576a966bd5a878b7ee8389074bf45014

Remap high-bitdepth functions to latest implementation.
Cleanup dead functions to avoid future implementations to rely on it.

Change-Id: I048c6e9cc790520247cc21ae9b92a9c8d84d00a7

BUG=aomedia:448

Change-Id: Ic41ec5c4bb321a611efb7e2ac3a96d6e30d88be1

BUG=aomedia:443

Change-Id: Iebdb609564c2c3040f3902052afeeec7ae83412d

Change-Id: Ie45f96c21f0276f464cad3c7f69ac3dceed50c67

Change-Id: Iaa8ab77eb4a982759939fb5fc475f699cb21a4e1

Rename '--enable-aom-highbitdepth' to '--enable-highbitdepth'

Change-Id: I1de13c3508c30c552532993419d8ace326142ab6

This function will apply av1_inverse_transform_block to
pd->dst.buf directly.

Change-Id: I703762c2d3cee2af626c190fe3b3995e3cce2082

Change-Id: I35b2b69e3937e70a7923ba76735f035f366de27f

Move most of the code of inverse_transform_block to
av1_inverse_transform_block such that encoder can use the
function as well.

Use av1_inverse_transform_block in av1_encoder_block_intra.
This will make the code cleaner.

Change-Id: I84dbeef2c65e252910606dbea446ce43165d504c

Change-Id: If7ccca58c816edd09e5ffec1b0bc97c5ac6bee97

Change-Id: Id7cbb89ba344770297ef42bf1b09a2b9ba5a1378

Change-Id: I73e3eec0b8fd17c3f9b9f52afc9fac43f3043028

This commit fixes the 2x2 transform system setups for high bit-
depth setting. It enables the cb4x4 mode to support high bit-depth
process. The coding performance is improved over high bit-depth +
ref-mv:

lowres  2.5%
midres  1.2%

Change-Id: I351f9d72bdc7e15b2bd00e94286b98966a295e6d

Add 2x2 forward and inverse 2D-DCT for high bit-depth.

Change-Id: I3092a2587a0cdc6675a69cc9203499a530b65325

Change-Id: Ic4775ed7e9470230deda2d8f656d94e3f81df99a

Scale down the 2x2 inverse transform transfer gain by 2.

Change-Id: Ifdd2ed302c05d08be142bc2fdef276f176e38c3d

This function will be used in the 4x4 coding block unit.

Change-Id: I981c2abce4ae1e1571adb0d87c76e2307e94300b

Also includes some refactoring and cleanups.

Change-Id: I2c2528c434a1e9e9b898251fa69489d884463929

For higher level fwd and inv transform functions.

Change-Id: I91518250a0be7d94aada7519f6c9e7ed024574fb

Change-Id: Ief08b23b30b78d640f6d7c702145e5bcf1b37b57

Change-Id: I213f3111fc0656aecd1303a8b871ecded2b92bc2

This prepares the integration of rectangular transform block size
with recursive transform block partition system.

Change-Id: Id96aa3790dace15619c665f438241938992d1730

- Add unit tests to verify the bit-exact result.
- User level time reduction (EXT_TX):
    encoder: 3.63%
    decoder: 2.36%
- Also add tx_type=V_DCT...H_FLIPADST SSE2 for 16x16 inv txfm.

Change-Id: Idc6d9e8254aa536e5f18a87fa0d37c6bd551c083

Make linking to libvpx and libaom at the same time possible.

Change-Id: I7bab8527a32e446e3d564e6fa5d94ccd056bc63f