If a stream starts with a non-zero granpos, we would subtract 80 ms
 from the target time, and then report failure when the best page
 we could find (the first page in the link) had a granpos larger
 than that.
This worked for normal streams because the subtraction would fail
 (granpos can't wrap past -1) and we checked for that.