Skip to content
  • Luc Trudeau's avatar
    [CFL] Fixed point implementation for tx average · bfe2827b
    Luc Trudeau authored
    This change does not impact the bitstream as no loss is incured by using
    a fixed point value for the transform size average.
    
    For low bit depth, the transform size average is stored using Q8.10
    fixed point format. Worst case, smallest fraction is 1/1024.
    
    Results on Subset1 (Compared to 366b74 with CfL)
    
      PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
    0.0000 |  0.0000 |  0.0000 |   0.0000 | 0.0000 |  0.0000 |     0.0000
    
    Change-Id: Ia5b046b92a0e4c40e413b16af3394bdc0a8c8cd9
    bfe2827b