• Jingning Han's avatar
    Fix overflow issue in SSSE3 32x32 quantization · abff6788
    Jingning Han authored
    The 32x32 quantization process can potentially have the intermediate
    stacks over 16-bit range, thereby causing enc/dec mismatch. This commit
    fixes this overflow issue in the SSSE3 implementation, as well as the
    prototype, of 32x32 quantization.
    This fixes issue 607 from webm@googlecode.
    Change-Id: I85635e6ca236b90c3dcfc40d449215c7b9caa806