draft-ietf-avt-rtp-vorbis-04

Vorbis is a general purpose perceptual audio codec intended to allow maximum encoder flexibility, thus allowing it to scale competitively over an exceptionally wide range of bitrates. At the high quality/bitrate end of the scale (CD or DAT rate stereo, 16/24 bits), it is in the same league as AAC. Vorbis is also intended for lower and higher sample rates (from 8kHz telephony to 192kHz digital masters) and a range of channel representations (monaural, polyphonic, stereo, quadraphonic, 5.1, ambisonic, or up to 255 discrete channels). Vorbis encoded audio is generally encapsulated within an Ogg format bitstream , which provides framing and synchronization. For the purposes of RTP transport, this layer is unnecessary, and so raw Vorbis packets are used in the payload.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 .

For RTP based transport of Vorbis encoded audio the standard RTP header is followed by a 4 octets payload header, then the payload data. The payload headers are used to associate the Vorbis data with its associated decoding codebooks as well as indicating if the following packet contains fragmented Vorbis data and/or the number of whole Vorbis data frames. The payload data contains the raw Vorbis bitstream information. There are 3 types of Vorbis payload data, an RTP packet MUST contain just one of them at a time.

The format of the RTP header is specified in and shown in Figure . This payload format uses the fields of the header in a manner consistent with that specification.

The RTP header begins with an octet of fields (V, P, X, and CC) to support specialized RTP uses (see and for details). For Vorbis RTP, the following values are used. Version (V): 2 bits This field identifies the version of RTP. The version used by this specification is two (2). Padding (P): 1 bit Padding MAY be used with this payload format according to section 5.1 of . Extension (X): 1 bit The Extension bit is used in accordance with . CSRC count (CC): 4 bits The CSRC count is used in accordance with . Marker (M): 1 bit Set to zero. Audio silence suppression not used. This conforms to section 4.1 of . Payload Type (PT): 7 bits An RTP profile for a class of applications is expected to assign a payload type for this format, or a dynamically allocated payload type SHOULD be chosen which designates the payload as Vorbis. Sequence number: 16 bits The sequence number increments by one for each RTP data packet sent, and may be used by the receiver to detect packet loss and to restore packet sequence. This field is detailed further in . Timestamp: 32 bits A timestamp representing the sampling time of the first sample of the first Vorbis packet in the RTP packet. The clock frequency MUST be set to the sample rate of the encoded audio data and is conveyed out-of-band (e.g. as a SDP parameter). SSRC/CSRC identifiers: These two fields, 32 bits each with one SSRC field and a maximum of 16 CSRC fields, are as defined in .

The 4 octets following the RTP Header section are the Payload Header. This header is split into a number of bitfields detailing the format of the following payload data packets.

Ident: 24 bits This 24 bit field is used to associate the Vorbis data to a decoding Configuration. It is stored as network byte order integer. Fragment type (F): 2 bits This field is set according to the following list 0 = Not Fragmented 1 = Start Fragment 2 = Continuation Fragment 3 = End Fragment Vorbis Data Type (VDT): 2 bits This field specifies the kind of Vorbis data stored in this RTP packet. There are currently three different types of Vorbis payloads. Each packet MUST contain only a single type of Vorbis payload (e.g. you MUST not aggregate configuration and comment payload in the same packet) 0 = Raw Vorbis payload 1 = Vorbis Packed Configuration payload 2 = Legacy Vorbis Comment payload 3 = Reserved The packets with a VDT of value 3 MUST be ignored The last 4 bits represent the number of complete packets in this payload. This provides for a maximum number of 15 Vorbis packets in the payload. If the packet contains fragmented data the number of packets MUST be set to 0.

Raw Vorbis packets are currently unbounded in length, application profiles will likely define a practical limit. Typical Vorbis packet sizes range from very small (2-3 bytes) to quite large (8-12 kilobytes). The reference implementation typically produces packets less than ~800 bytes, except for the setup header packets which are ~4-12 kilobytes. Within an RTP context, to avoid fragmentation, the Vorbis data packet size SHOULD be kept sufficiently small so that after adding the RTP and payload headers, the complete RTP packet is smaller than the path MTU.

Each Vorbis payload packet starts with a two octet length header, which is used to represent the size in bytes of the following data payload, followed by the raw Vorbis data padded to the nearest byte boundary, as explained by the vorbis specification. The length value is stored as network byte order integer. For payloads which consist of multiple Vorbis packets the payload data consists of the packet length followed by the packet data for each of the Vorbis packets in the payload. The Vorbis packet length header is the length of the Vorbis data block only and does not count the length field. The payload packing of the Vorbis data packets MUST follow the guidelines set-out in where the oldest packet occurs immediately after the RTP packet header. Subsequent packets, if any, MUST follow in temporal order. Channel mapping of the audio is in accordance with the Vorbis I Specification.

Here is an example RTP packet containing two Vorbis packets. RTP Packet Header:

The payload data section of the RTP packet begins with the 24 bit Ident field followed by the one octet bitfield header, which has the number of Vorbis frames set to 2. Each of the Vorbis data frames is prefixed by the two octets length field. The Packet Type and Fragment Type are set to 0. The Configuration that will be used to decode the packets is the one indexed by the ident value.

Unlike other mainstream audio codecs Vorbis has no statically configured probability model. Instead, it packs all entropy decoding configuration, Vector Quantization and Huffman models into a data block that must be transmitted to the decoder along with the compressed data. A decoder also requires information detailing the number of audio channels, bitrates and similar information to configure itself for a particular compressed data stream. These two blocks of information are often referred to collectively as the "codebooks" for a Vorbis stream, and are nominally included as special "header" packets at the start of the compressed data. In addition, the Vorbis I specification requires the presence of a comment header packet which gives simple metadata about the stream, but this information is not required for decoding the frame sequence. Thus these two codebook header packets must be received by the decoder before any audio data can be interpreted. These requirements pose problems in RTP, which is often used over unreliable transports. Since this information must be transmitted reliably and, as the RTP stream may change certain configuration data mid-session, there are different methods for delivering this configuration data to a client, both in-band and out-of-band which is detailed below. SDP delivery is typically used to set up an initial state for the client application. The changes may be due to different codebooks as well as different bitrates of the stream. The delivery vectors in use can be specified by an SDP attribute to indicate the method and the optional URI where the Vorbis Packed Configuration Packets could be fetched. Different delivery methods MAY be advertised for the same session. The in-band Configuration delivery SHOULD be considered as baseline, out-of-band delivery methods that don't use RTP will not be described in this document. For non chained streams, the Configuration recommended delivery method is inline the Packed Configuration in the SDP as explained in the IANA considerations. The 24 bit Ident field is used to map which Configuration will be used to decode a packet. When the Ident field changes, it indicates that a change in the stream has taken place. The client application MUST have in advance the correct configuration and if the client detects a change in the Ident value and does not have this information it MUST NOT decode the raw Vorbis data associated until it fetches the correct Configuration.

The Packed Configuration Payload is sent in-band with the packet type bits set to match the Vorbis Data Type. Clients MUST be capable of dealing with fragmentation and periodic re-transmission of the configuration headers.

A Vorbis Packed Configuration is indicated with the Vorbis Data Type field set to 1. Of the three headers, defined in the Vorbis I specification, the identification and the setup MUST be packed together as they are, while the comment header MAY be replaced with a dummy one. The packed configuration follows a generic way to store xiph codec configurations: the first byte stores the number of following packets minus one, the next bytes represent the size of the packets, every byte set to 0xff means that the next byte has to be add to the current sum in order to have the complete size; the headers immediately follows the list of sizes. The headers are packed in the same order they are present in ogg: identification, comment, setup.

The Ident field is set with the value that will be used by the Raw Payload Packets to address this Configuration. The Fragment type is set to 0 since the packet bears the full Packed configuration, the number of packet is set to 1.

This section, as stated above, does not cover all the possible out-of-band delivery methods since they rely on different protocols and are linked to specific applications. The following packet definition SHOULD be used in out-of-band delivery and MUST be used when Configuration is inlined in the SDP.

As mentioned above the RECOMMENDED delivery vector for Vorbis configuration data is via a retrieval method that can be performed using a reliable transport protocol. As the RTP headers are not required for this method of delivery the structure of the configuration data is slightly different. The packed header starts with a 32 bit (network ordered) count field which details the number of packed headers that are contained in the bundle. Next is the Packed header payload for each chained Vorbis stream.

Since the Configuration Ident and the Identification Header are fixed length there is only a 2 byte length tag to define the length of the packed headers.

The key difference between the in-band format and this one, is there is no need for the payload header octet. In this figure the comment has a size bigger than 255 bytes.

Unlike the loss of raw Vorbis payload data, loss of a configuration header can lead to a situation where it will not be possible to successfully decode the stream. Loss of Configuration Packet results in the halting of stream decoding.

With the Vorbis Data Type flag set to 2, this indicates that the packet contain the comment metadata, such as artist name, track title and so on. These metadata messages are not intended to be fully descriptive but to offer basic track/song information. Clients MAY ignore it completely. The details on the format of the comments can be found in the Vorbis documentation.

The 2 bytes length field is necessary since this packet could be fragmented.

Each RTP packet contains either one Vorbis packet fragment, or an integer number of complete Vorbis packets (up to a maximum of 15 packets, since the number of packets is defined by a 4 bit value). Any Vorbis data packet that is less than path MTU SHOULD be bundled in the RTP packet with as many Vorbis packets as will fit, up to a maximum of 15, except when such bundling would exceed an application's desired transmission latency. Path MTU is detailed in and . A fragmented packet has a zero in the last four bits of the payload header. The first fragment will set the Fragment type to 1. Each fragment after the first will set the Fragment type to 2 in the payload header. The RTP packet containing the last fragment of the Vorbis packet will have the Fragment type set to 3. To maintain the correct sequence for fragmented packet reception the timestamp field of fragmented packets MUST be the same as the first packet sent, with the sequence number incremented as normal for the subsequent RTP packets. The length field shows the fragment length.

Here is an example fragmented Vorbis packet split over three RTP packets. Each packet contains the standard RTP headers as well as the 4 octets Vorbis headers.

In this packet the initial sequence number is 1000 and the timestamp is 12345. The Fragment type is set to 1, the number of packets field is set to 0, and as the payload is raw Vorbis data the VDT field is set to 0.

The Fragment type field is set to 2 and the number of packets field is set to 0. For large Vorbis fragments there can be several of these type of payload packets. The maximum packet size SHOULD be no greater than the path MTU, including all RTP and payload headers. The sequence number has been incremented by one but the timestamp field remains the same as the initial packet.

This is the last Vorbis fragment packet. The Fragment type is set to 3 and the packet count remains set to 0. As in the previous packets the timestamp remains set to the first packet in the sequence and the sequence number has been incremented.

As there is no error correction within the Vorbis stream, packet loss will result in a loss of signal. Packet loss is more of an issue for fragmented Vorbis packets as the client will have to cope with the handling of the Fragment Type. In case of loss of fragments the client MUST discard all the remaining fragments and decode the incomplete packet. If we use the fragmented Vorbis packet example above and the first packet is lost the client MUST detect that the next packet has the packet count field set to 0 and the Fragment type 2 and MUST drop it. The next packet, which is the final fragmented packet, MUST be dropped in the same manner. If the missing packet is the last, the received two fragments will be kept and the incomplete vorbis packet decoded. Loss of any of the Configuration fragment will result in the loss of the full Configuration packet with the result detailed in the Loss of Configuration Headers section.

audio vorbis indicates the RTP timestamp clock rate as described in RTP Profile for Audio and Video Conferences with Minimal Control. indicates the number of audio channels as described in RTP Profile for Audio and Video Conferences with Minimal Control. indicates the delivery methods in use, the possible values are: inline, in_band, out_band, MAY be included multiple times the base64 representation of the Packed Headers. It MUST follow the associated delivery-method parameter ("inline"). the URI of the configuration headers in case of out of band transmission. In the form of "protocol://path/to/resource/". Depending on the specific method, a single configuration packet could be retrived by its number, or multiple packets could be aggregated in a single stream. Such aggregates MAY be compressed using either bzip2 or gzip. A sha1 checksum MAY be provided for aggregates. In this latter case the URI will end with the aggregate name, followed by its compressed extension if applies, a "!" and the base64 representation of the sha1hash of the above mentioned compressed aggregated as in: "protocol://path/to/resource/aggregated.bz2!sha1hash". The trailing '/' discriminates which of two methods are in use. It MUST follow the associated delivery method parameter (either "in_band" or "out_band"). This media type is framed and contains binary data. See Section 10 of RFC XXXX. None RFC XXXX [RFC Editor: please replace by the RFC number of this memo, when published] Ogg Vorbis I specification: Codec setup and packet decode. Available from the Xiph website, http://www.xiph.org Audio streaming and conferencing tools None Luca Barbato: <lu_zero@gentoo.org>
IETF Audio/Video Transport Working Group COMMON This media type depends on RTP framing, and hence is only defined for transfer via RTP Luca Barbato IETF AVT Working Group delegated from the IESG

The following IANA considerations MUST only be applied to the packed headers. audio vorbis-config None None This media type contains binary data. See Section 10 of RFC XXXX. None RFC XXXX [RFC Editor: please replace by the RFC number of this memo, when published] Vorbis encoded audio, configuration data. None Luca Barbato: <lu_zero@gentoo.org> IETF Audio/Video Transport Working Group COMMON This media type doesn't depend on the transport. Luca Barbato IETF AVT Working Group delegated from the IESG

The following paragraphs defines the mapping of the parameters described in the IANA considerations section and their usage in the Offer/Answer Model.

The information carried in the Media Type media type specification has a specific mapping to fields in the Session Description Protocol (SDP), which is commonly used to describe RTP sessions. When SDP is used to specify sessions the mapping are as follows: The type name ("audio") goes in SDP "m=" as the media name. The subtype name ("vorbis") goes in SDP "a=rtpmap" as the encoding name. The parameter "rate" also goes in "a=rtpmap" as clock rate. The parameter "channels" also goes in "a=rtpmap" as channel count. The mandated parameters "delivery-method" and "configuration" MUST be included in the SDP "a=fmtp" attribute. The optional parameter "configuration-uri", when present, MUST be included in the SDP "a=fmtp" attribute and MUST follow the delivery-method that applies. If the stream comprises chained Vorbis files and all of them are known in advance, the Configuration Packet for each file SHOULD be passed to the client using the configuration attribute. The URI specified in the configuration-uri attribute MUST point to a location where all of the Configuration Packets needed for the life of the session reside. The port value is specified by the server application bound to the address specified in the c= line. The bitrate value and channels specified in the rtpmap attribute MUST match the Vorbis sample rate value. An example is found below.

The following example shows a basic SDP single stream. The first configuration packet is inlined in the sdp, other configurations could be fetched at any time from the first provided uri using or all the known configuration could be downloaded using the second uri. The inline base64 configuration string is omitted because of the length. c=IN IP4 192.0.2.1 m=audio RTP/AVP 98 a=rtpmap:98 vorbis/44100/2 a=fmtp:98 delivery-method=in_band; configuration=base64string; delivery-method=out_band; configuration-uri=rtsp://path/to/the/resource; delivery-method=out_band; configuration-uri=http://another/path/to/resource/aggregate.bz2!8b6237eb5154a0ea12811a94e8e2697b3312bc6c;

Note that the payload format (encoding) names are commonly shown in upper case. MIME subtypes are commonly shown in lower case. These names are case-insensitive in both places. Similarly, parameter names are case-insensitive both in MIME types and in the default mapping to the SDP a=fmtp attribute. The exception regarding case sensitivity is the configuration-uri URI which MUST be regarded as being case sensitive. The a=fmtp line is a single line even if it is presented broken because of clarity.

The only paramenter negotiable is the delivery method. All the others are declarative: the offer, as described in An Offer/Answer Model Session Description Protocol, may contain a large number of delivery methods per single fmtp attribute, the answerer MUST remove every delivery-method and configuration-uri not supported. All the parameters MUST not be altered on answer otherwise.

Vorbis clients SHOULD send regular receiver reports detailing congestion. A mechanism for dynamically downgrading the stream, known as bitrate peeling, will allow for a graceful backing off of the stream bitrate. This feature is not available at present so an alternative would be to redirect the client to a lower bitrate stream if one is available.

The following examples are common usage patterns that MAY be applied in such situations, the main scope of this section is to explain better usage of the transmission vectors.

This is one of the most common situation: one single server streaming content in multicast, the clients may start a session at random time. The content itself could be a mix of live stream, as the wj's voice, and stored streams as the music she plays. In this situation we don't know in advance how many codebooks we will use. The clients can join anytime and users expect to start listening to the content in a short time. On join the client will receive the current Configuration necessary to decode the current stream inlined in the SDP so that the decoding will start immediately after. When the streamed content changes the new Configuration is sent in-band before the actual stream, and the Configuration that has to be sent inline in the SDP updated. Since the in-band method is unreliable, an out of band fallback is provided. The client could choose to fetch the Configuration from the alternate source as soon as it discovers a Configuration packet got lost in-band or use selective retransmission, if the server supports the feature. A serverside optimization would be to keep an hash list of the Configurations per session to avoid packing all of them and send the same Configuration with different Ident tags A clientside optimization would be to keep a tag list of the Configurations per session and don't process configuration packets already known.

RTP packets using this payload format are subject to the security considerations discussed in the RTP specification . This implies that the confidentiality of the media stream is achieved by using encryption. Because the data compression used with this payload format is applied end-to-end, encryption may be performed on the compressed data. Additional care MAY be needed for delivery methods that point to external resources, using secure protocols to fetch the configuration payloads. Where the size of a data block is set, care MUST be taken to prevent buffer overflows in the client applications.

This document is a continuation of draft-moffitt-vorbis-rtp-00.txt and draft-kerr-avt-vorbis-rtp-04.txt. The MIME type section is a continuation of draft-short-avt-rtp-vorbis-mime-00.txt. Thanks to the AVT, Ogg Vorbis Communities / Xiph.org including Steve Casner, Aaron Colwell, Ross Finlayson, Fluendo, Ramon Garcia, Pascal Hennequin, Ralph Giles, Tor-Einar Jarnbjo, Colin Law, John Lazzaro, Jack Moffitt, Christopher Montgomery, Colin Perkins, Barry Short, Mike Smith, Phil Kerr, Michael Sparks, Magnus Westerlund, David Barrett, Silvia Pfeiffer, Stefan Ehmann, Alessandro Salvatori. Politecnico di Torino (LS)�/IMG Group in particular Federico Ridolfo, Francesco Varano, Giampaolo Mancini, Juan Carlos De Martin.