Commit 1a8664ea authored by Debargha Mukherjee's avatar Debargha Mukherjee

Do not use length-64 transform for chroma

Adds a missing logic in get_vartx_max_txsize() function for
64x16 and 16x64 transforms.

Change-Id: I60bf4f5b49be674f103e30a2e35fa0a43ba1f7e6
parent 780fad1a
......@@ -1244,7 +1244,12 @@ static INLINE int get_vartx_max_txsize(const MACROBLOCKD *xd, BLOCK_SIZE bsize,
// mustn't be used for the subsampled plane (because it would be bigger than
// a 64x64 luma block) so we round down to TX_32X32.
if (subsampled && txsize_sqr_up_map[max_txsize] == TX_64X64) {
max_txsize = TX_32X32;
if (max_txsize == TX_16X64)
max_txsize = TX_16X32;
else if (max_txsize == TX_64X16)
max_txsize = TX_32X16;
else
max_txsize = TX_32X32;
}
#else
(void)subsampled;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment