building with TLS causes shout_set_metadata to fail with SHOUTERR_INSANE with Shoutcast servers
Related to #2244 (closed) -- tested with latest libshout master (2dd6cfb7) on OSX.
Mixxx 2.0 and onwards uses libshout 2.4.1 built with TLS enabled (using SHOUT_TLS_AUTO) on OSX and we were receiving user reports of metadata not working with Shoutcast (not Icecast) servers.
I saw ph3-der-loewe's patch: https://github.com/xiph/Icecast-libshout/commit/4542dc2d7efd7b12ab80a45cfa4bb4ff6d03fc24 which I thought would fix it, but I can still reproduce at head.
I dug in and realized that shout_set_metadata was returning SHOUTERR_INSANE because self->tls_mode_used is SHOUTERR_NOTLS at the time we send metadata. My understanding is that if tls_mode is SHOUT_TLS_AUTO, then the correct result of the state machine is that self->tls_mode_used becomes SHOUT_TLS_DISABLED if a TLS upgrade is not possible.
Here is the Mixxx bug which has more details: https://bugs.launchpad.net/mixxx/+bug/1544739
I applied this hack to try_connect where it says "TODO: do something":
++ if (self->tls_mode == SHOUT_TLS_AUTO_NO_PLAIN) {
++ self->tls_mode_used = SHOUTERR_NOTLS;
++ return SHOUTERR_NOTLS;
++ }
++ self->tls_mode_used = SHOUT_TLS_DISABLED;
which works around the issue.
I doubt this is the right fix though since I think this block is supposed to "poke the server" like the comment says above :) since we haven't successfully probed the server yet.