• Timothy B. Terriberry's avatar
    encoder: Remove 64x upsampled reference buffers · 5d24b6f0
    Timothy B. Terriberry authored
    They do not handle border extension correctly (interpolation and
    border extension do not commute unless you upsample into the
    border), nor do they handle crop dimensions that are not a multiple
    of 8 (the upsampled version is not sufficiently large), in addition
    to using massive amounts of memory and being a criminal waste of
    cache (1 byte used for every 8 bytes fetched).
    
    This commit reimplements use_upsampled_references by computing the
    subpixel samples on the fly. This implementation not only corrects
    the border handling, but is also faster, while maintaining the
    same quality.
    
    HL AWCY results are basically noise:
        PSNR | PSNR HVS |   SSIM | MS SSIM | CIEDE 2000
      0.0188 |   0.0187 | 0.0045 |  0.0063 |     0.0228
    
    Change-Id: I7527db9f83b87a7bb8b35342f7e6457cd0bef9cd
    5d24b6f0
variance.c 55.8 KB