• Angie Chiang's avatar
    Remove saturate_int16 from fdct_round_shift · 48c06da2
    Angie Chiang authored
    1) Not every transform's internal signal is designed to fit in 16 bits.
    2) If overflow happens in this function, it indicates that we need to
    adjust the txfm's scaling. We shouldn't mute the overflow signal.
    3) Saturation might be handy when all of our transform design are stable,
    but I don't think we are at the stable point yet.
    4) This will fix C/Trans16x16DCT.AccuracyCheck/1 failure in highbd mode.
    
    Change-Id: I5ef5d130c22adb4b8c3b608ffcb0f2c99dc7523f
    48c06da2
fwd_txfm.h 1.03 KB