AVX2 runtime detection not working properly
I'm building opus 1.5.2 with the following cmake options:
-DOPUS_X86_MAY_HAVE_AVX2=on -DOPUS_X86_PRESUME_AVX2=off -DOPUS_DEEP_PLC=on
The build process is done on a machine with AVX2. When I run my application on a machine without AVX2 it crashes in silk_inner_product_FLP_avx2 with "trap invalid opcode".
This lets me presume that AVX2 runtime detection does not work properly.
Activity
Platform:
linux amd64
Compiler
gcc (Debian 13.2.0-25) 13.2.0
Errors via linux dmesg:
[ 1281.171143] traps: grpcpp_sync_ser[12770] trap invalid opcode ip:55f9a9fee85c sp:7f53c57f2b78 error:0
Errors from gdb:
#0 0x000055beca8b72dc in silk_inner_product_FLP_avx2 (data1=0x7f156a7f4ca0, data2=0x7f156a7f4ca0, dataSize=384) at silk/float/x86/inner_product_FLP_avx2.c:50
The machine, it was build on:
Vendor ID: GenuineIntel Model name: Intel Core Processor (Broadwell, no TSX) CPU family: 6 Model: 61 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 Stepping: 2 BogoMIPS: 4800.00 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pcl mulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault pti ssbd ibrs ibpb tpr_shado w flexpriority ept vpid ept_ad fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap xsaveopt arat vnmiThe machine, it was run and crashed:
Vendor ID: GenuineIntel BIOS Vendor ID: QEMU Model name: Intel(R) Xeon(R) Gold 6161 CPU @ 2.20GHz BIOS Model name: pc-i440fx-2.8 CPU family: 6 Model: 85 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 4 BogoMIPS: 4400.00 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 hle smep bmi2 erms invpcid rtm rdseed adx smap clflushopt clwb arat md_clear flush_l1d `` Need more information?Edited by flo-eyeson-teamRunning sde it seems to work as expected: ./../../sde-external-9.38.0-2024-04-18-lin/sde64 -ivb -- ./test_opus_encode ...
./../../sde-external-9.38.0-2024-04-18-lin/sde64 -- ./test_opus_encode Testing libopus 1.5.2-11-g2554a89e encoder. Random seed: 269938783 (80D8) Running simple tests for bugs that have been fixed previously silk_inner_product_FLP_avx2 silk_inner_product_FLP_avx2 silk_inner_product_FLP_avx2 silk_inner_product_FLP_avx2
So do I get it right, that it seems that the run-time detection detects AVX2 because the cpuid reports so, because the cpu model indeed supports avx2. But finally the QEMU-Virtualization does not support avx2 execution, right?
If so, should the opus runtime check be improved to honor lscpu feature flags or is the cloud-hoster not playing by the rules?
I tried with list_cpu_features (https://github.com/google/cpu_features) and it does not detect AVX2 on that machine.
Any more information needed?
I have certainly run into issues before with QEMU and confusion between the host and the emulation target's capabilities. cpu_features also uses CPUID, though. The only thing I can think of that it does differently is that it also checks to see if the OS saves YMM registers (which was not possible to do in an OS-agnostic way before AVX). libopus currently does not do that.
Are you able to confirm that this is the difference on your system? I.e., can you look at the value of have_xcr0 and os_preserves->avx_registers determined in ParseCpuId()?
Seems to be the case. Both values are false:
have_xcr0: 0
os_preserves->avx_registers: 0
[~]$ ./list_cpu_features have_xcr0: 0 os_preserves->avx_registers: 0 arch : x86 brand : Intel(R) Xeon(R) Gold 6161 CPU @ 2.20GHz family : 6 (0x06) model : 85 (0x55) stepping : 4 (0x04) uarch : INTEL_SKL flags : adx,aes,bmi1,bmi2,clflushopt,clfsh,clwb,cx16,cx8,erms,f16c,fpu,hle,lzcnt,mmx,movbe,pclmulqdq,popcnt,rdrnd,rdseed,rtm,ss,sse,sse2,sse3,sse4_1,sse4_2,ssse3,tsc cache_info : {"level":1,"cache_type":"data","cache_size":32768,"ways":8,"line_size":64,"tlb_entries":64,"partitioning":1},{"level":1,"cache_type":"instruction","cache_size":32768,"ways":8,"line_size":64,"tlb_entries":64,"partitioning":1},{"level":2,"cache_type":"unified","cache_size":1048576,"ways":16,"line_size":64,"tlb_entries":1024,"partitioning":1},{"level":3,"cache_type":"unified","cache_size":31719424,"ways":11,"line_size":64,"tlb_entries":45056,"partitioning":1}Edited by flo-eyeson-teamHi, I've hit this bug as well trying to run a VM with qemu. Here's the stacktrace I get:
Core was generated by `/usr/bin/qemu-system-x86_64 -name guest=windows11,debug-threads=on -S -object {'. Program terminated with signal SIGILL, Illegal instruction. Downloading source file /usr/src/debug/opus/build/../opus-1.5.2/celt/x86/pitch_avx.c #0 0x00006aa8348196ff in celt_pitch_xcorr_avx2 (arch=<optimized out>, max_pitch=<optimized out>, len=<optimized out>, xcorr=<optimized out>, _y=<optimized out>, _x=<optimized out>) at ../opus-1.5.2/celt/x86/pitch_avx.c:97 97 xcorr[i] = celt_inner_prod(_x, _y+i, len, arch); [Current thread is 1 (Thread 0x6aa838aed380 (LWP 23910))] (gdb) bt #0 0x00006aa8348196ff in celt_pitch_xcorr_avx2 (arch=<optimized out>, max_pitch=<optimized out>, len=<optimized out>, xcorr=<optimized out>, _y=<optimized out>, _x=<optimized out>) at ../opus-1.5.2/celt/x86/pitch_avx.c:97 #1 celt_pitch_xcorr_avx2 (_x=0x7cb19e86abf0, _y=<optimized out>, xcorr=0x7cb19e86aba0, len=748, max_pitch=5, arch=<optimized out>) at ../opus-1.5.2/celt/x86/pitch_avx.c:86 #2 0x00006aa8348627a7 in _celt_autocorr.isra.0 (x=x@entry=0x7cb19e86abf0, ac=ac@entry=0x7cb19e86aba0, window=window@entry=0x0, overlap=overlap@entry=0, lag=lag@entry=4, n=n@entry=752, arch=<optimized out>) at ../opus-1.5.2/celt/celt_lpc.c:333 #3 0x00006aa834823298 in pitch_downsample (x=x@entry=0x7cb19e86fc90, x_lp=x_lp@entry=0x7cb19e86abf0, len=len@entry=1504, C=C@entry=2, arch=<optimized out>) at ../opus-1.5.2/celt/pitch.c:184 #4 0x00006aa83482b5e0 in run_prefilter (analysis=<optimized out>, nbAvailableBytes=<optimized out>, enabled=<optimized out>, qgain=<synthetic pointer>, gain=<synthetic pointer>, pitch=<synthetic pointer>, prefilter_tapset=<optimized out>, N=<optimized out>, CC=<optimized out>, prefilter_mem=<optimized out>, in=<optimized out>, st=<optimized out>) at ../opus-1.5.2/celt/celt_encoder.c:1221 #5 celt_encode_with_ec (st=<optimized out>, pcm=<optimized out>, frame_size=<optimized out>, compressed=<optimized out>, nbCompressedBytes=<optimized out>, enc=<optimized out>) at ../opus-1.5.2/celt/celt_encoder.c:1694 #6 0x00006aa8348106ab in opus_encode_frame_native (st=st@entry=0x366b4d64030, pcm=pcm@entry=0x7cb19e876f30, frame_size=frame_size@entry=480, data=<optimized out>, data@entry=0x366b43923e0 "\364\377\376", max_data_bytes=max_data_bytes@entry=480, float_api=float_api@entry=0, first_frame=<optimized out>, dred_bitrate_bps=<optimized out>, analysis_info=<optimized out>, is_silence=<optimized out>, redundancy=<optimized out>, celt_to_silk=<optimized out>, prefill=<optimized out>, equiv_rate=<optimized out>, to_celt=<optimized out>) at ../opus-1.5.2/src/opus_encoder.c:2299 #7 0x00006aa83481471f in opus_encode_native (st=0x366b4d64030, pcm=<optimized out>, frame_size=<optimized out>, data=<optimized out>, out_data_bytes=<optimized out>, lsb_depth=<optimized out>, lsb_depth@entry=16, analysis_pcm=<optimized out>, analysis_size=<optimized out>, c1=<optimized out>, c2=<optimized out>, analysis_channels=<optimized out>, downmix=<optimized out>, float_api=<optimized out>) at ../opus-1.5.2/src/opus_encoder.c:1682 #8 0x00006aa834815931 in opus_encode (st=<optimized out>, pcm=<optimized out>, analysis_frame_size=480, data=<optimized out>, max_data_bytes=<optimized out>) at ../opus-1.5.2/src/opus_encoder.c:2517 #9 0x00006aa834e90c9b in snd_codec_encode_opus (out_size=<synthetic pointer>, out_ptr=0x366b43923e0 "\364\377\376", in_size=1920, in_ptr=0x366b439572c "", codec=<optimized out>) at ../spice-0.15.2/subprojects/spice-common/common/snd_codec.c:108 #10 snd_codec_encode (out_size=<synthetic pointer>, out_ptr=0x366b43923e0 "\364\377\376", in_size=1920, in_ptr=0x366b439572c "", codec=<optimized out>) at ../spice-0.15.2/subprojects/spice-common/common/snd_codec.c:244 #11 snd_playback_send_write (playback_client=0x366b4391bc0) at ../spice-0.15.2/server/sound.cpp:591 #12 PlaybackChannelClient::send_item (this=0x366b4391bc0, item=<optimized out>) at ../spice-0.15.2/server/sound.cpp:658 #13 0x00006aa834e76d6a in RedChannelClient::send_any_item (item=0x366b4391be0, --Type <RET> for more, q to quit, c to continue without paging--c this=0x366b4391bc0) at ../spice-0.15.2/server/red-channel-client.cpp:513 #14 RedChannelClient::push (this=0x366b4391bc0) at ../spice-0.15.2/server/red-channel-client.cpp:1164 #15 0x00006aa834e9051d in RedChannelClient::pipe_add_push (item=..., this=<optimized out>) at ../spice-0.15.2/server/red-channel-client.cpp:1398 #16 snd_send (client=0x366b4391bc0) at ../spice-0.15.2/server/sound.cpp:636 #17 0x00006aa8384a5395 in line_out_put_buffer () at ../qemu-9.1.0/audio/spiceaudio.c:160 #18 0x0000036682a9bda9 in audio_pcm_hw_run_out () at ../qemu-9.1.0/audio/audio.c:1117 #19 audio_run_out () at ../qemu-9.1.0/audio/audio.c:1226 #20 0x0000036682a9c55d in audio_run () at ../qemu-9.1.0/audio/audio.c:1388 #21 audio_timer () at ../qemu-9.1.0/audio/audio.c:855 #22 0x0000036682f1566b in timerlist_run_timers () at ../qemu-9.1.0/util/qemu-timer.c:576 #23 0x0000036682f15800 in timerlist_run_timers () at ../qemu-9.1.0/util/qemu-timer.c:509 #24 qemu_clock_run_timers () at ../qemu-9.1.0/util/qemu-timer.c:590 #25 qemu_clock_run_all_timers () at ../qemu-9.1.0/util/qemu-timer.c:677 #26 0x0000036682f15aa3 in main_loop_wait () at ../qemu-9.1.0/util/main-loop.c:600 #27 0x0000036682e3f327 in qemu_main_loop () at ../qemu-9.1.0/system/runstate.c:826 #28 qemu_default_main () at ../qemu-9.1.0/system/main.c:37 #29 0x00006aa8393fae08 in __libc_start_call_main (main=main@entry=0x3668283ab90 <main>, argc=argc@entry=139, argv=argv@entry=0x7cb19e8782e8) at ../sysdeps/nptl/libc_start_call_main.h:58 #30 0x00006aa8393faecc in __libc_start_main_impl (main=0x3668283ab90 <main>, argc=139, argv=0x7cb19e8782e8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7cb19e8782d8) at ../csu/libc-start.c:360 #31 0x000003668283dc35 in _start ()This is not the first time I report such an issue, AVX detection is hard to get right. I use a hardened kernel which disables it to mitigate a security vulnerability in my CPU, even though it is supported.
You can take a look at this orc MR which fixed the issue some time ago.
In case that matters:
$ pacman -Qi opus Name : opus Version : 1.5.2-1 Description : Totally open, royalty-free, highly versatile audio codec Architecture : x86_64 URL : https://www.opus-codec.org/ Licenses : BSD-3-Clause Groups : None Provides : libopus.so=0-64 Depends On : glibc Optional Deps : None Required By : chromium ffmpeg gst-plugins-base libsndfile pipewire-audio spice spice-gtk wireshark-cli Optional For : None Conflicts With : None Replaces : None Installed Size : 5.30 MiB Packager : Jan Alexander Steffens (heftig) <heftig@archlinux.org> Build Date : Wed 17 Apr 2024 02:57:01 AM CEST Install Date : Tue 30 Apr 2024 12:15:16 AM CEST Install Reason : Installed as a dependency for another package Install Script : No Validated By : Signature$ uname -a Linux laptop 6.10.12-hardened1-1-hardened #1 SMP PREEMPT_DYNAMIC Wed, 02 Oct 2024 17:47:30 +0000 x86_64 GNU/LinuxEdited by ShellCode
