• Luc Trudeau's avatar
    [CFL] Fewer bits for fixed point · 475fc9df
    Luc Trudeau authored
    Since alpha is Q3, we reduce y_average from Q10 to Q3. As such, the
    prediction is reduced from Q13 to Q6. Chroma dc_pred is reduced from Q7
    to Q6 in order to match with the prediction.
    
    Results on Subset1 (compared to 209de2e5b with CfL enabled)
    
      PSNR | PSNR Cb | PSNR Cr | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
    0.0010 |  0.0176 | -0.0538 |  -0.0043 | 0.0027 | -0.0097 |    -0.0018
    
    Change-Id: Ib7dd3968a764e0380ddc0ad2333ebacf1e9699cd
    475fc9df
cfl.h 3.26 KB