-
Dominic Symes authored
Coefficients used for the VP9 version of IADST4 satisfy the mathematical relation sinpi[1]+sinpi[2]==sinpi[4] which is a trigonometric property useful for optimizing the four point transform. Unfortunately the change in shift used in the latest code means that rounding errors are introduced so that this identity no longer holds. We think the identity should be restored by rounding 4964.563 to 4964 rather than 4965. Similarly the identity has been fixed for other bit shifts in the table where it does not hold (even though these are not currently used I think). This change also fixes a bug with the range checking for IADST4. We see little change on the statistics from a local ACWY run on objective-1-fast I-frame only: PSNR PSNR-HVS SSIM PSNR-Cr APSNR APSNR-Cr MS-SSIM PSRNR-YUV Average +0.01 +0.01 -0.02 -0.15 +0.01 -0.15 +0.01 +0.01 BUG=aomedia:1360 Change-Id: Icc09a1c59929e58bbc922d4b8b73de4a14104a8e
6f81f128