Commit e0f3a756 authored by Sebastien Alaiwan's avatar Sebastien Alaiwan

inv_txfm accuracy test: use right cos_bit precision

Match cos_bit precision actually used in the decoder (max: 13).

Change-Id: I609f59a9653d74b569420bb0d402ba4f5352aa76
parent 69f2adaa
......@@ -42,7 +42,7 @@ const TxfmFunc inv_txfm_func_ls[][2] = {
};
// the maximum stage number of fwd/inv 1d dct/adst txfm is 12
const int8_t cos_bit[12] = { 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14 };
const int8_t cos_bit[12] = { 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13 };
const int8_t range_bit[12] = { 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32 };
TEST(av1_inv_txfm1d, round_trip) {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment