Commit ab1cad9b authored by Yaowu Xu's avatar Yaowu Xu
Browse files

fix a small bug in 16 point forward dct

The commit fixes a minor error in 16 point fdct where in a rotation can
produce result of -1 instead of 0.

Change-Id: I45aac4a52bcd06225c6d04e643547a13e1c1aade
parent 5149d7f7
......@@ -1288,7 +1288,7 @@ static void dct16x16_1d(int16_t input[16], int16_t output[16],
output[15] = (intermediate[11] + intermediate[12] + output_rounding)
>> output_shift;
output[ 1] = -(intermediate[11] - intermediate[12] + output_rounding)
output[ 1] = (intermediate[12] - intermediate[11] + output_rounding)
>> output_shift;
output[ 7] = (2 * (intermediate[13] * C8) + final_rounding) >> final_shift;
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment