• John Koleszar's avatar
    Fix drastic undershoot in long form content · 36be4f7f
    John Koleszar authored
    When the modified_error_left accumulator exceeds INT_MAX, an incorrect
    cast to int resulted in a negative value, causing the rate control to
    allocate no bits to that keyframe group, leading to severe undershoot
    and subsequent poor quality.
    This error was exposed by the recent change to the rolling target and
    actual spend accumulators in commit 305be4e4 which fixed them to
    actually calculate the average value rather than be re-initialized
    on every frame to the average per-frame bitrate. When this bug was
    triggered, the target bitrate could be 0, so the rolling target
    becomes small, which causes the undershoot. The code prior to 305be4e4
    did not exhibit this behavior because the rolling target was always
    set to a reasonable value and was independent of the actual target
    bitrate. With this patch, the actual target bitrate is calculated
    correctly, and the rate control tracks as expected.
    This cast was likely added to silence a compiler warning on a comparison
    between a double (modified_error_left) and an int (0). Instead, this
    patch removes the cast and changes the comparison to be against 0.0,
    which should prevent the warning from reoccuring.
    This fixes issue #289. Special thanks to gnafu for his efforts in
    reporting and debugging this fix.
    Change-Id: Ie5cc1a7b516c578a76c3a50c892a6f04a11621fe
firstpass.c 110 KB