Skip to content
  • Ronald S. Bultje's avatar
    Consistently use get_prob(), clip_prob() and newly added clip_pixel(). · 4d0ec7aa
    Ronald S. Bultje authored
    Add a function clip_pixel() to clip a pixel value to the [0,255] range
    of allowed values, and use this where-ever appropriate (e.g. prediction,
    reconstruction). Likewise, consistently use the recently added function
    clip_prob(), which calculates a binary probability in the [1,255] range.
    If possible, try to use get_prob() or its sister get_binary_prob() to
    calculate binary probabilities, for consistency.
    
    Since in some places, this means that binary probability calculations
    are changed (we use {255,256}*count0/(total) in a range of places,
    and all of these are now changed to use 256*count0+(total>>1)/total),
    this changes the encoding result, so this patch warrants some extensive
    testing.
    
    Change-Id: Ibeeff8d886496839b8e0c0ace9ccc552351f7628
    4d0ec7aa