aom-qm: Fix overflow in quantizer calculation
By using a combination of a small qmlevel and large qindex, it is possible for the quantizer 'dqv' to end up > 2^15. This currently causes dqv to overflow, so widen its type to avoid that. Note that this is certainly an extreme case, but not entirely useless - if using a 32x32 transform, the dequantized value can be as small as dqv/2, which may still fit in 16 bits. Change-Id: I0451e70ba6b17620d94386a98d873f0a8c615635
Please register or sign in to comment