• Yi Luo's avatar
    Fix inv txfm low/high bitdepth selection logic · 51281095
    Yi Luo authored
    We are going to have several commits to setup new low/high
    bitdepth data path selection logic. This patch is for inverse
    transform. Let me summarize the ideas as following.
    - For low/high bitdepth selection, encoder depends on
      input configuration, e.g., video sequence bitdepth,
      profile. Decoder depends on input bitstream. This has
      nothing to do with compiler/build  configuration.
    - Typical encoder usage for sampling format 4:2:0.
      1) 8-bit video sequence:
       a) --profile=0
       Fastest encoding/decoding pipeline on speedup.
       b) --profile=2 --bit-depth=10
       Image pixels are left shifted by 2 bits. It
       employs 16-bit reference frame buffer and has high
       calculation precision. It usually enjoys higher
       compression performance.
      2) 10/12-bit video sequence (HDR):
       --profile=2 --bit-depth=10/12
    - Transform coefficient type:
      Lowbitdepth:  int16_t
      Highbitdepth: int32_t
    - The type, tran_low_t is still used in codebase,
      Which is int32_t, defining the data path capacity.
      Naturally, it is high bitdepth.
    Eventually we shall remove the configuration flags,
    low and high bitdepth data path. Two data paths co-exist
    in the same build environment.
    Change-Id: I35c06d4d4f19ebf80d909168fdddbae57c3cc884