Commit 66d94ac1 authored by Yaowu Xu's avatar Yaowu Xu
Browse files

Improve 32x32 forward dct

The commit improves the 32x32 forward dct implementation:
1. change to use same constants and rounding as other forward dcts
2. select rounding to specifically minimize the roundtrip error, which
improved average 19/block to .77/block using 100000 random input.

Test showed a small but consistent gain on all test sets, about .15%

Change-Id: If0afd6a71880a522f60c1c234be0462092c2eb53
parent 9770d564
......@@ -61,4 +61,11 @@ static INLINE int dct_const_round_shift(int input) {
assert(INT16_MIN <= rv && rv <= INT16_MAX);
return rv;
}
static INLINE int dct_32_round(int input) {
int rv = (input + DCT_CONST_ROUNDING) >> DCT_CONST_BITS;
assert(-131072 <= rv && rv <= 131071);
return rv;
}
#endif
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment