Skip to content
  • David Barker's avatar
    Fix integer overflow in warp filter · 17c37ceb
    David Barker authored
    Patch https://aomedia-review.googlesource.com/c/12602/ made the
    variable 'sum' in the warp filter unsigned, to indicate that its
    value should always be >= 0. But 'sum' is used to accumulate
    signed values, and it is expected that some of those values
    will be negative.
    
    The issue is that, when running 'x += y', if x is a uint32_t
    and y is an int (and is 32 bits), the C standard says to
    convert y to a uint32_t before doing the addition. This causes
    overflow, and so undefined behaviour, if y < 0.
    
    This is fixed by making 'sum' signed, and by explicitly bounds
    checking against zero at the end of the filter.
    
    BUG=aomedia:572
    
    Change-Id: I1d484b5f5698db0ec9761807610b3b2b35647983
    17c37ceb