• Monty's avatar
    floor0 code could potentially use a book where the number of vals it · 80661a13
    Monty authored
    needed to decode was not an integer number of dims wide.  This caused
    it to overflow the output vector as the termination condition was in
    the outer loop of vorbis_book_decodev_set.
    None of the various vorbis_book_decodeXXXX calls internally guard
    against this case either, but in every other use the calling code does
    properly guard (and avoids putting more checks in the tight inner
    decode loop).
    For floor0, move the checks into the inner loop as there's little
    penalty for doing so.  Add commentary indicating where guarding is
    done for each call variant.
    git-svn-id: https://svn.xiph.org/trunk/Tremor@17546 0101bb08-14d6-0310-b084-bc0e0c8e3800
codebook.c 10.5 KB