-
David Barker authored
In both versions of av1_optimize_b, the code to calculate the dequantized coefficient from a modified quantized coefficient was not compatible with tx64x64. Specifically, the new dequantized coefficient was calculated as an offset applied to the original coefficient. This had a fixup block for 32x32 transforms, but not for 64x64 transforms. For simplicity, instead of extending the fixup block, we now recalculate the dequantized coefficient directly. BUG=aomedia:537 Change-Id: I86c1c8820f324ad3f6afd6a720ff59f754384a19
c747a78e