• Ronald S. Bultje's avatar
    32x32 transform for superblocks. · c456b35f
    Ronald S. Bultje authored
    This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds
    code all over the place to wrap that in the bitstream/encoder/decoder/RD.
    
    Some implementation notes (these probably need careful review):
    - token range is extended by 1 bit, since the value range out of this
      transform is [-16384,16383].
    - the coefficients coming out of the FDCT are manually scaled back by
      1 bit, or else they won't fit in int16_t (they are 17 bits). Because
      of this, the RD error scoring does not right-shift the MSE score by
      two (unlike for 4x4/8x8/16x16).
    - to compensate for this loss in precision, the quantizer is halved
      also. This is currently a little hacky.
    - FDCT and IDCT is double-only right now. Needs a fixed-point impl.
    - There are no default probabilities for the 32x32 transform yet; I'm
      simply using the 16x16 luma ones. A future commit will add newly
      generated probabilities for all transforms.
    - No ADST version. I don't think we'll add one for this level; if an
      ADST is desired, transform-size selection can scale back to 16x16
      or lower, and use an ADST at that level.
    
    Additional notes specific to Debargha's DWT/DCT hybrid:
    - coefficient scale is different for the top/left 16x16 (DCT-over-DWT)
      block than for the rest (DWT pixel differences) of the block. Therefore,
      RD error scoring isn't easily scalable between coefficient and pixel
      domain. Thus, unfortunately, we need to compute the RD distortion in
      the pixel domain until we figure out how to scale these appropriately.
    
    Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
    c456b35f
vp9_invtrans.h 1.57 KB