Fix crashes with VBR for short duration frames and very low bitrates. This...

Fix crashes with VBR for short duration frames and very low bitrates. This may, however, cause the encoder to violate the rate target at insanely low rates.

This also generally improves VBR behavior by more carefully accounting
for rounding.