Commit e0413094 authored by Jingning Han's avatar Jingning Han

Add horizontal and vertical scan order for 1-D transform

This commit enables the 1-D transform to use Manhattan grid vertical
and horizontal scan order for transform coefficient entropy coding.

Enabled in inter prediction mode, the hybrid 1D/2D transform coding
scheme outperforms the 2D-DCT based coding system used in VP9 by
lowres_all  1.7%
hdres_all   1.4%

As one coding option, in addition to the existing 17 other transform
types in ext-tx experiment, the 1D/2D hybrid transform improves
the coding gains:
lowres_all  2.2% -> 3.0%

Change-Id: I9cefa9d9e38224546d0afd67feecd9f8d4a16ab0
parent a8dc9694
This diff is collapsed.
......@@ -292,10 +292,15 @@ static void inverse_transform_block_inter(MACROBLOCKD* xd, int plane,
} else {
if (tx_type == DCT_DCT && tx_size <= TX_16X16 && eob <= 10)
memset(dqcoeff, 0, 4 * (4 << tx_size) * sizeof(dqcoeff[0]));
#if CONFIG_EXT_TX
else
memset(dqcoeff, 0, (16 << (tx_size << 1)) * sizeof(dqcoeff[0]));
#else
else if (tx_size == TX_32X32 && eob <= 34)
memset(dqcoeff, 0, 256 * sizeof(dqcoeff[0]));
else
memset(dqcoeff, 0, (16 << (tx_size << 1)) * sizeof(dqcoeff[0]));
#endif
}
}
}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment