Invalid memory access with samplingrate==0
I am forwarding this issue from the Debian bug tracker: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=716613
The Mayhem software detected a crash when passing a specially crafted FLAC file to oggenc. I've examined the problem further and found out that the problem is that the flac reader provides an input stream with a sampling rate of 0. This is something that neither oggenc itself nor libvorbisenc can cope with. The oggenc executable crashes with SIGFPE (division by the sampling rate in the statistics => division by 0) and libvorbisenc accesses invalid memory during the initialization.
While the oggenc crash doesn't affect the usability much (there is nothing valid to encode anyway in a file with sampling rate 0), libvorbisenc's access to invalid memory has the potential to either crash the whole downstream application with SIGSEGV or even be a security issue (although my preliminary examination didn't show any sign of exploitability).
The latest versions of vorbis-tools and libvorbis (and probably many earlier versions) are affected (I'm reporting this here because these version numbers don't appear in the TRAC version dropdown):
- libvorbis: 1.3.4
- vorbis-tools: 1.4.0
Please find attached the patches that we used in Debian. They add sanity checks to both the oggenc executable and the initialization routines of libvorbis.