OSUOSL/Nero are experiencing Internet connectivity problems. This affects us as we're hosted with OSUOSL. We apologize for the inconvenience.

Commit 09191ca2 authored by Urvang Joshi's avatar Urvang Joshi

Reduce cos bits for 64x64.

This is to get the high range within 32 bits.

The compression performance is neutral.

Change-Id: I475b2694f7cc7d786478c8821cf17982281dabce
parent c502216a
......@@ -115,9 +115,9 @@ static const int8_t fwd_stage_range_row_dct_64[12] = { 6, 7, 8, 9, 10, 11,
static const int8_t fwd_stage_range_idx_64[1] = { 0 };
// cos bit
static const int8_t fwd_cos_bit_col_dct_64[12] = { 15, 15, 15, 15, 15, 14,
static const int8_t fwd_cos_bit_col_dct_64[12] = { 13, 13, 13, 13, 13, 13,
13, 13, 13, 13, 13, 13 };
static const int8_t fwd_cos_bit_row_dct_64[12] = { 15, 14, 13, 12, 11, 10,
static const int8_t fwd_cos_bit_row_dct_64[12] = { 13, 13, 13, 12, 11, 10,
10, 10, 10, 10, 10, 10 };
// ---------------- row config fwd_dct_4 ----------------
......
......@@ -179,12 +179,10 @@ int bd_arr[BD_NUM] = { 8, 10, 12 };
#if CONFIG_TX64X64
int8_t low_range_arr[BD_NUM] = { 18, 32, 32 };
// TODO(urvang): Try reducing cos bit by 1 for TX64X64 to get it back to 32.
int8_t high_range_arr[BD_NUM] = { 33, 33, 33 };
#else
int8_t low_range_arr[BD_NUM] = { 16, 32, 32 };
int8_t high_range_arr[BD_NUM] = { 32, 32, 32 };
#endif // CONFIG_TX64X64
int8_t high_range_arr[BD_NUM] = { 32, 32, 32 };
void txfm_stage_range_check(const int8_t *stage_range, int stage_num,
const int8_t *cos_bit, int low_range,
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment