• David Barker's avatar
    Improve rectangular transform accuracy · 838a53d6
    David Barker authored
    By adjusting the internal scaling and rounding in the transforms,
    we can adjust the maximum round-trip errors to:
    * 8x16 and 16x8: 0 pixel values (ie, transforms are exact)
    * 16x32: 1 pixel value
    * 32x16: 2 pixel values
    
    Change-Id: I0ba691a8d27042dcf1dd5ae81568d07a92d68781
    838a53d6
dct.c 63.6 KB