Commit 48c06da2 authored by Angie Chiang's avatar Angie Chiang Committed by Yaowu Xu

Remove saturate_int16 from fdct_round_shift

1) Not every transform's internal signal is designed to fit in 16 bits.
2) If overflow happens in this function, it indicates that we need to
adjust the txfm's scaling. We shouldn't mute the overflow signal.
3) Saturation might be handy when all of our transform design are stable,
but I don't think we are at the stable point yet.
4) This will fix C/Trans16x16DCT.AccuracyCheck/1 failure in highbd mode.

Change-Id: I5ef5d130c22adb4b8c3b608ffcb0f2c99dc7523f
parent 54e24517
......@@ -22,7 +22,7 @@ static INLINE tran_high_t saturate_int16(tran_high_t value) {
static INLINE tran_high_t fdct_round_shift(tran_high_t input) {
tran_high_t rv = ROUND_POWER_OF_TWO(input, DCT_CONST_BITS);
return saturate_int16(rv);
return rv;
}
void aom_fdct32(const tran_high_t *input, tran_high_t *output, int round);
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment