Commit 0d43bd77 authored by Yunqing Wang's avatar Yunqing Wang

Bug fix in ssse3 quantize function

A bug was reported in Issue 702: "SIGILL (Illegal instruction) when
transcoding with vp9 - using FFmpeg". It was reproduced and fixed.

Change-Id: Ie32c149a89af02856084aeaf289e848a905c7700
parent 4f8a30b1
......@@ -188,7 +188,8 @@ cglobal quantize_%1, 0, %2, 15, coeff, ncoeff, skip, zbin, round, quant, \
pmaxsw m8, m7
pshuflw m7, m8, 0x1
pmaxsw m8, m7
pextrw [r2], m8, 0
pextrw r6, m8, 0
mov [r2], r6
RET
; skip-block, i.e. just write all zeroes
......@@ -214,5 +215,5 @@ cglobal quantize_%1, 0, %2, 15, coeff, ncoeff, skip, zbin, round, quant, \
%endmacro
INIT_XMM ssse3
QUANTIZE_FN b, 6
QUANTIZE_FN b, 7
QUANTIZE_FN b_32x32, 7
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment