Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

This stores the caps array in 32nd bits/sample instead of 1/2 bits
 scaled by LM and the channel count, which is slightly less
 less accurate for the last two bands, and much more accurate for
 all the other bands.
A constant offset is subtracted to allow it to represent values
 larger than 255 in 8 bits (the range of unoffset values is
 77...304).
In addition, this replaces the last modeline in the allocation table
 with the caps array, allowing the initial interpolation to
 allocate 8 bits/sample or more, which was otherwise impossible.

We did no real error checking to see if a mode is supported when it
 is created.
This patch implements checks for Jean-Marc's rules:
1) A mode must have frames at least 1ms in length (no more than
    1000 per second).
2) A mode must have shorts of at most 3.33 ms (at least 300 per
    second).
It also adds error checking to dump_modes so we report the error
 instead of crashing when we fail to create a mode.

The way folding is implemented requires two restrictions:
1. The last band must be the largest (so we can use its size to
 allocate a temporary buffer to handle interleaving/TF changes).
2. No band can be larger than twice the size of the previous band
 (so that once we have enough data to start folding, we will always
 have enough data to fold).

Mode creation makes a heuristic attempt to satisfy these
 conditions, but nothing actually guarantees it.
This adds some asserts to check them during mode creation.
They current pass for all supported custom modes.

Currently compute_ebands()'s attempts to round bands to even sizes
 and enforce size constraints on consecutive bands can leave some
 bands entirely empty (e.g., Fs=8000, frame_size=64, i=11).
This adds a simple post-processing loop to remove such bands.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

9b34bd83 caused serious regressions for 240-sample frame stereo,
 because the previous qb limit was _always_ hit for two-phase
 stereo.
Two-phase stereo really does operate with a different model (for
 example, the single bit allocated to the side should really
 probably be thought of as a sign bit for qtheta, but we don't
 count it as part of qtheta's allocation).
The old code was equivalent to a separate two-phase offset of 12,
 however Greg Maxwell's testing demonstrates that 16 performs
 best.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

Previously, we would only split a band if it was allocated more than
 32 bits.
However, the N=4 codebook can only produce about 22.5 bits, and two
 N=2 bands combined can only produce 26 bits, including 8 bits for
 qtheta, so if we wait until we allocate 32, we're guaranteed to fall
 short.
Several of the larger bands come pretty far from filling 32 bits as
 well, though their split versions will.

Greg Maxwell also suggested adding an offset to the threshold to
 account for the inefficiency of using qtheta compared to another
 VQ dimension.
This patch uses 1 bit as a placeholder, as it's a clear
 improvement, but we may adjust this later after collecting data on
 more possibilities over more files.

The first version of the mono decoder with stereo output collapsed
 the historic energy values stored for anti-collapse down to one
 channel (by taking the max).
This means that a subsequent switch back would continue on using
 the the maximum of the two values instead of the original history,
 which would make anti-collapse produce louder noise (and
 potentially more pre-echo than otherwise).

This patch moves the max into the anti_collapse function itself,
 and does not store the values back into the source array, so the
 full stereo history is maintained if subsequent frames switch
 back.
It also fixes an encoder mismatch, which never took the max
 (assuming, apparently, that the output channel count would never
 change).

Instead of just dumping excess bits into the first band after
 allocation, use them to initialize the rebalancing loop in
 quant_all_bands().
This allows these bits to be redistributed over several bands, like
 normal.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

The average caps over all values of LM and C are well below the
 target allocations of the last two modelines.
Lower them to the caps, to prevent hitting them quite so early.
This helps quality at medium-high rates, in the 180-192 kbps range.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

Use measured cross-entropy to estimate the real cost of coding
 qtheta given the allocated qb parameter, instead of the entropy of
 the PDF.
This is generally much lower, and reduces waste at high rates.
This patch also removes some intermediate rounding from this
 computation.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

This extends the previous rebalancing for fine energy in N=1 bands
 to also allocate extra fine bits for bands that go over their cap.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

The previous "dumb cap" of (64<<LM)*(C<<BITRES) was not actually
 achievable by many (most) bands, and did not take the cost of
 coding theta for splits into account, and so was too small for some
 bands.
This patch adds code to compute a fairly accurate estimate of the
 real maximum per-band rate (an estimate only because of rounding
 effects and the fact that the bit usage for theta is variable),
 which is then truncated and stored in an 8-bit table in the mode.

This gives improved quality at all rates over 160 kbps/channel,
 prevents bits from being wasted all the way up to 255 kbps/channel
 (the maximum rate allowed, and approximately the maximum number of
 bits that can usefully be used regardless of the allocation), and
 prevents dynalloc and trim from producing enormous waste
 (eliminating the need for encoder logic to prevent this).

We use the MDCT as low-pass filter.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

Previously, in a stereo split with itheta==16384, but without
 enough bits left over to actually code a pulse, the target band
 would completely collapse, because the mid gain would be zero and
 we don't fold the side.
This changes the limit to ensure that we never set qn>1 unless we
 know we'll have enough bits for at least one pulse.
This should eliminate the last possible whole-band collapse.

The old constructor is renamed celt_encoder_create_custom(). Same
for the decoder.

Gregory Maxwell authored 14 years ago and Jean-Marc Valin committed 14 years ago

Prevent VBR from shooting up to the maximum rate if set to very low target rates, and prevent the encoder VBR from producing 1 byte frames (which are no longer allowed).

The energy memory can be lowered (not increased) during a transient

That way they can be exact in 16 bits once multiplied by the gain