NaNs should be filtered at the Opus layer, so if there are any in the CELT
encoder, then it's likely something went horribly wrong (e.g. corrupted state).
In that case, better abort than have something bad happen.