pickmode: Fix a pair of unsigned overflows.
block_variance: This operates on 8x8s and would be safe with a int32 * int32 to uint32 multiply, but this is potentially unsafe for 12-bit input. Unfortunately the code already segfaults on 12-bit input: https://bugs.chromium.org/p/webm/issues/detail?id=1223 calculate_variance: This operates on up to a 32x32 of 8x8s and can overflow even with 8-bit input (log2((256*32*32)**2) == 36). BUG=https://bugs.chromium.org/p/webm/issues/detail?id=1220 Change-Id: I1ca4ff6092db9a7580da371ee9a21f403fdadc40
Showing with 2 additions and 2 deletions