opus_decode LP PLC fails to accommodate frame size change during packet loss
Suppose opus packets are being received over an unreliable network with a frame size and packet size of 40ms. There is some packet loss, and during that time the sender switches to 10ms packets, as shown below:
Received packets:
| = timestamp = | = content = |
|---|---|
| //t// | 40ms frame |
| //t//+80ms | 10ms frame |
opus_decode() is called in the usual manner for the time //t// packet. When the next packet is received for time //t//+80ms, opus_decode() is called with that packet using decode_fec=1 and frame_size=1920 (40ms), to fill in the gap from //t//+40ms to //t//+80ms.
If libopus assertions are enabled, an assertion failure is produced:
Fatal (internal) error in silk/dec_API.c, line 142: assertion failed: 0
Abort trapIf libopus assertions are disabled it does not abort, but the pcm output contains 30ms of digital silence from //t//+40ms to //t//+70ms before the PLC (or FEC if available) activates. This is shown in the attached picture, in which //t//=0 and the audio is a continuous 880 Hz sine wave with constant amplitude.
It expected that the entire gap from //t//+40ms to //t//+80ms will be filled in with interpolated PLC data and any available FEC data, as it does with no frame size change, without producing an assertion failure.
Note that the same type of failure also occurs for other times and frame sizes, e.g. if the second packet contains a 2.5ms MDCT frame for time //t//+55ms.
The issue can be reproduced by decoding the attached opus_demo bitstream.
opus_demo -d 48000 1 -inbandfec opus-lp-plc-fail.opusdemo out.pcm