Thanks for the quick response @phschafft, I assumed you were ignoring this, sorry for the the rudeness. You didn't deserve that.
I wasn't able to capture a stacktrace last time, and before that was pure laziness, just not wanting to deal with this. My fault.
Would you recommend I try the latest main brain checkout, or your personal branch @phschafft ?
Yep, completely gave up @MJLeeming. Right now we just have a script that we run manually that renews the cert and does a service icecast2 restart
, which boots all our listeners off. Less than ideal. Even now an icecast2 reload
is insufficient to prevent this bug, despite a commit a while back that addressed TLS issues.
Contrary to what the Icecast developers suggest, we're going to be putting nginx in front of Icecast so we can stop dealing with this and just let the nginx LetsEncrypt plugin do its work automatically, so we never have to worry again.
@phschafft this issue should not have been closed.
phschafft has taught me on IRC to get stacktraces and straces as such:
[11:29:43 AM] <phschafft> you can get a stracelog by 'strace -s 1024 -fp $PID -o icecast.stracelog' (-o is the output filename). keep it running for a few seconds and then ^C to quit strace. Be aware, the strace log can/will include information about what is going in including client addresses or parts of the config.
[11:30:04 AM] <phschafft> top get a backtrace: attach gdb with: gdb -p $PID
[11:30:09 AM] <phschafft> then in gdb: thread apply all bt
[11:30:34 AM] <phschafft> save the result. you can then quit gdb with: quit
[11:30:49 AM] <phschafft> it may ask you if it should kill the process or keep it running.
Using the latest git version of Icecast, every month or two my Icecast server stops responding to HTTP requests and e.g. kill -hup
/ kill
signals.
With the logs being set to WARN
, there is nothing particular in the access or error logs. I've set the logs to DEBUG
now for the next time in hopes of having some more insight.
Some more details:
I'm using the latest git so that I can SIGHUP
the server when I refresh my LetsEncrypt certificates, but the timing of these reloads seems completely disconnected from the server hanging. The timing, at this point, seems random.
I will update this ticket the next time this happens, hopefully with log details.
Please instruct me if there's anything I can do to probe the issue on my end, such as enable particular compile flags for debugging, etc. Also please instruct me if there's any further information you need, such as distribution, GCC version, etc.