Commit 31bc29f6 authored by Timothy B. Terriberry's avatar Timothy B. Terriberry

cb4x4: Don't assume TX units are half MI_SIZE in predict_square_intra_block()

Currently the "transform units" used to measure transform sizes
in some parts of the code are based on the smallest defined
transform size.

cb4x4 currently defines a 2x2 transform size, even when chroma_2x2
is not enabled, which means that the scale of the transform units
was always double that of MODEINFO units.

Several areas of the code were hard-coding this assumption instead
of converting from one to the other using appropriate constants.

Change-Id: I86057b85b7d4cd810d87da28f5eedc5b27597e29
parent a2d5cdef
......@@ -2420,9 +2420,9 @@ static void predict_square_intra_block(const MACROBLOCKD *xd, int wpx, int hpx,
// the frame bottom edge
const int yd = (xd->mb_to_bottom_edge >> (3 + pd->subsampling_y)) +
(hpx - y - txhpx) - yd_chr_offset;
const int right_available =
(mi_col + ((col_off + txw) >> (1 - pd->subsampling_x))) <
xd->tile.mi_col_end;
const int right_available = mi_col + ((col_off + txw) << pd->subsampling_x >>
(MI_SIZE_LOG2 - tx_size_wide_log2[0])) <
xd->tile.mi_col_end;
const int bottom_available = (yd > 0);
#if CONFIG_EXT_PARTITION_TYPES
const PARTITION_TYPE partition = xd->mi[0]->mbmi.partition;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment