• Jingning Han's avatar
    Fix ioc in trellis optimization with hbd · 07d35de0
    Jingning Han authored
    Use int64_t type for distortion. This avoids integer overflow
    issues in the trellis optimization function in high bit-depth
    settings.
    
    Change-Id: I550c3ca9f11a3191ef8638a152887018cd476141
    07d35de0
encodemb.c 49 KB