• Ronald S. Bultje's avatar
    Prevent filling transform size cache with uninitialized values. · ebfdaa0e
    Ronald S. Bultje authored
    The 32x32 value in case of splitmv was uninitialized. this leads to
    all kind of erratic behaviour down the line. Also fill in dummy values
    for superblocks in keyframes (the values are currently unused, but we
    run into integer overflows anyway, which makes detecting bad cases
    harder). Lastly, in case we did not find any RD value at all, don't
    set tx_diff to INT_MIN, but instead set it to zero (since if we couldn't
    find a mode, it's unlikely that any particular transform would have made
    that worse or better; rather, it's likely equally bad for all tx_sizes).
    
    Change-Id: If236fd3aa2037e5b398d03f3b1978fbbc5ce740e
    ebfdaa0e
vp9_rdopt.c 172 KB