Skip to content
  • Geza Lore's avatar
    Optimize vp9_highbd_block_error_8bit assembly. · aa8f8522
    Geza Lore authored
    A new version of vp9_highbd_error_8bit is now available which is
    optimized with AVX assembly. AVX itself does not buy us too much, but
    the non-destructive 3 operand format encoding of the 128bit SSEn integer
    instructions helps to eliminate move instructions. The Sandy Bridge
    micro-architecture cannot eliminate move instructions in the processor
    front end, so AVX will help on these machines.
    
    Further 2 optimizations are applied:
    
    1. The common case of computing block error on 4x4 blocks is optimized
    as a special case.
    2. All arithmetic is speculatively done on 32 bits only. At the end of
    the loop, the code detects if overflow might have happened and if so,
    the whole computation is re-executed using higher precision arithmetic.
    This case however is extremely rare in real use, so we can achieve a
    large net gain here.
    
    The optimizations rely on the fact that the coefficients are in the
    range [-(2^15-1), 2^15-1], and that the quantized coefficients always
    have the same sign as the input coefficients (in the worst case they are
    0). These are the same assumptions that the old SSE2 assembly code for
    the non high bitdepth configuration relied on. The unit tests have been
    updated to take this constraint into consideration when generating test
    input data.
    
    Change-Id: I57d9888a74715e7145a5d9987d67891ef68f39b7
    aa8f8522