Commit 403618eb authored by Yushin Cho's avatar Yushin Cho

Fix the condition of recode when PVQ enabled

The definition of TX_SIZES has changed after nextgenv2 is merged,
which included non-square transforms. TX_SIZES is used as part of
condition doing recode in tx_size/type RDO when PVQ is enabled.

Change-Id: I9a66a72f0f4391eb375290aa0dabc638c68c6c68
parent 0e11912a
......@@ -1772,9 +1772,8 @@ static void choose_tx_size_type_from_rd(const AV1_COMP *const cpi,
if (mbmi->tx_size >= TX_32X32) assert(mbmi->tx_type == DCT_DCT);
#endif
#if CONFIG_PVQ
if (best_tx < TX_SIZES) {
RD_STATS this_rd_stats;
txfm_rd_in_plane(x, cpi, &this_rd_stats, ref_best_rd, 0, bs, best_tx,
if (best_rd != INT64_MAX) {
txfm_rd_in_plane(x, cpi, rd_stats, ref_best_rd, 0, bs, best_tx,
cpi->sf.use_fast_coef_costing);
}
#endif
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment