CVE-2017-11333 - The vorbis_analysis_wrote function in lib/block.c in Xiph.Org libvorbis 1.3.5 allows remote attackers to cause a denial of service (OOM) via a crafted wav file.

Discovered by qflb.wu See 2nd issue in http://seclists.org/fulldisclosure/2017/Jul/82

Copy paste from there:

1.
the vorbis_analysis_wrote function in lib/block.c in Xiph.Org libvorbis 1.3.5 can cause a denial of service(OOM) via a 
crafted wav file.


I found this bug when I test Sound eXchange(SoX) 14.4.2 which used the libvorbis library.


./sox libvorbis_1.3.5_OOM.wav out.ogg


/var/log/syslog info:
  Jul 13 19:58:05 ubuntu kernel: [] Out of memory: Kill process 44203 (sox) score 364 or sacrifice child
Jul 13 19:58:05 ubuntu kernel: [] Killed process 44203 (sox) total-vm:1831804kB, anon-rss:599932kB, file-rss:40kB


----debug info:----
#0  0x00007ffff5df5e92 in vorbis_analysis_wrote ()
from /usr/local/lib/libvorbis.so.0
#1  0x00007ffff7ba1cba in write_samples (ft=0x611c20, buf=buf@entry=0x0, 
    len=len@entry=0x0) at vorbis.c:358
#2  0x00007ffff7ba1dc5 in stopwrite (ft=<optimized out>) at vorbis.c:398
#3  0x00007ffff7b58488 in sox_close (ft=0x611c20) at formats.c:1006
#4  0x0000000000405fa8 in cleanup () at sox.c:246
#5  0x0000000000403479 in main (argc=argc@entry=0x3, 
    argv=argv@entry=0x7fffffffe5e8) at sox.c:3050
#6  0x00007ffff727bec5 in __libc_start_main (main=0x4029c0 <main>, argc=0x3, 
    argv=0x7fffffffe5e8, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffe5d8) at libc-start.c:287
#7  0x0000000000403c65 in _start ()
--------
Program terminated with signal SIGKILL, Killed.
The program no longer exists.