• Jingning Han's avatar
    Fix high bit-depth quantization process · 6ae7b3d9
    Jingning Han authored
    Scale the rounding factor according to the scaling factor applied
    to the quantization step size. This resolves a compression
    performance regression in 32x32 and above transform size.
    
    BUG=aomedia:599
    
    Change-Id: Id3fc9a46c4a8843ff5d77ccaa59ee3112b12d7f4
    6ae7b3d9
av1_quantize.c 68.5 KB