Previously, recombining only worked when going all the way back to
frequency domain.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

The right amount of data was being written for the first frame, but
 from the wrong offset in the buffer.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

This patch makes all symbols conditional on whether or not there's
 enough space left in the buffer to code them, and eliminates much
 of the redundancy in the side information.

A summary of the major changes:
* The isTransient flag is moved up to before the the coarse energy.
  If there are not enough bits to code the coarse energy, the flag
   would get forced to 0, meaning what energy values were coded
   would get interpreted incorrectly.
  This might not be the end of the world, and I'd be willing to
   move it back given a compelling argument.
* Coarse energy switches coding schemes when there are less than 15
   bits left in the packet:
  - With at least 2 bits remaining, the change in energy is forced
     to the range [-1...1] and coded with 1 bit (for 0) or 2 bits
     (for +/-1).
  - With only 1 bit remaining, the change in energy is forced to
     the range [-1...0] and coded with one bit.
  - If there is less than 1 bit remaining, the change in energy is
     forced to -1.
    This effectively low-passes bands whose energy is consistently
     starved; this might be undesirable, but letting the default be
     zero is unstable, which is worse.
* The tf_select flag gets moved back after the per-band tf_res
   flags again, and is now skipped entirely when none of the
   tf_res flags are set, and the default value is the same for
   either alternative.
* dynalloc boosting is now limited so that it stops once it's given
   a band all the remaining bits in the frame, or when it hits the
   "stupid cap" of (64<<LM)*(C<<BITRES) used during allocation.
* If dynalloc boosing has allocated all the remaining bits in the
   frame, the alloc trim parameter does not get encoded (it would
   have no effect).
* The intensity stereo offset is now limited to the range
   [start...codedBands], and thus doesn't get coded until after
   all of the skip decisions.
  Some space is reserved for it up front, and gradually given back
   as each band is skipped.
* The dual stereo flag is coded only if intensity>start, since
   otherwise it has no effect.
  It is now coded after the intensity flag.
* The space reserved for the final skip flag, the intensity stereo
   offset, and the dual stereo flag is now redistributed to all
   bands equally if it is unused.
  Before, the skip flag's bit was given to the band that stopped
   skipping without it (usually a dynalloc boosted band).

In order to enable simple interaction between VBR and these
 packet-size enforced limits, many of which are encountered before
 VBR is run, the maximum packet size VBR will allow is computed at
 the beginning of the encoding function, and the buffer reduced to
 that size immediately.
Later, when it is time to make the VBR decision, the minimum packet
 size is set high enough to ensure that no decision made thus far
 will have been affected by the packet size.
As long as this is smaller than the up-front maximum, all of the
 encoder's decisions will remain in-sync with the decoder.
If it is larger than the up-front maximum, the packet size is kept
 at that maximum, also ensuring sync.
The minimum used now is slightly larger than it used to be, because
 it also includes the bits added for dynalloc boosting.
Such boosting is shut off by the encoder at low rates, and so
 should not cause any serious issues at the rates where we would
 actually run out of room before compute_allocation().

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

There were two different ones in use, one with less precision than
 a float, and the other missing a digit in the middle.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

B contains the number of blocks _after_ splitting.
We were using it to decide a) when to use a uniform PDF instead of a
 triangular one for theta and b) whether to bias the bit allocation
 towards the lower bins.
Using B0 (the number of blocks before the split) instead for a)
 gives a PEAQ gain of 0.003 ODG (as high as 0.1 ODG on s02a samples
 006, 083, and 097) for 240-sample frames at 96kbps mono.
Using B0 instead for b) gives a gain of only 0.00002.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

The mid = (lo+hi)>>1 line in the binary search would allow hi to drop
 down to the same value as lo, meaning the rounding after the search
 would be choosing between the same two values.
This patch changes it to (lo+hi+1)>>1.
This will allow lo to increase up to the value hi, but only in the
 case that we can't possibly allocate enough pulses to meet the
 target number of bits (in which case the rounding doesn't matter).
To pay for the extra add, this moves the +1 in the comparison to bits
 to the other side, which can then be taken outside the loop.
The compiler can't normally do this because it might cause overflow
 which would change the results.

This rarely mattered, but gives a 0.01 PEAQ improvement on 12-byte
 120 sample frames.
It also makes the search process describable with a simple
 algorithm, rather than relying on this particular optimized
 implementation.
I.e., the binary search loop can now be replaced with
  for(lo=0;lo+1<cache[0]&&cache[lo+1]<bits;lo++);
  hi=lo+1;
 and it will give equivalent results.
This was not true before.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

This renames ec_dec_cdf() to ec_dec_icdf(), and changes the
 functionality to use an "inverse" CDF table, where
 icdf[i]=ft-cdf[i+1].
The first entry is omitted entirely.
It also adds a corresonding ec_enc_icdf() to the encoder, which uses
 the same table.
One could use ec_encode_bin() by converting the values in the tables
 back to normal CDF values, but the icdf[] table already has them in
 the form ec_encode_bin() wants to use them, so there's no reason to
 translate them and then translate them back.

This is done primarily to allow SILK to use the range coder with
 8-bit probability tables containing cumulative frequencies that
 span the full range 0...256.
With an 8-bit table, the final 256 of a normal CDF becomes 0 in the
 "inverse" CDF.
It's the 0 at the start of a normal CDF which would become 256, but
 this is the value we omit, as it already has to be special-cased in
 the encoder, and is not used at all in the decoder.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

The band where intensity stereo begins was being coded as an
 absolute value, rather than relative to start, even though the
 range of values in the bitstream was limited as if it was being
 coded relative to start (meaning there would be desync if
 intensity was sufficiently large).

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

Previously it was coded for all bands, even when not all of them
 were being used.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

The valid bands range from [start,end) everywhere, with start<end.
Therefore end should never be 0, and should be allowed to extend
 all the way to mode->nbEBands.
This patch does _not_ enforce that start<end, and it does _not_
 handle clearing oldBandE[] when the valid range changes, which
 are separate issues.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

cf874373 raised the limit from 7 to 8 for N>1 bands in
 interp_bits2pulses(), but did not raise the corresponding limits
 for N=1 bands, or for [un]quant_energy_finalise().
This commit raises all of the limits to the same value, 8.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

This way if a band doesn't get the fine bits we want because it
 wasn't allocated enough bits to start with, then we will still
 give it priority for any spare bits after PVQ.

Compensate for the fact that the side "energy" is not preserved
when the split ends up starving one segment.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

The tf_res bits are useless without tf_select, so move it up before
 them.
This allows the tf_res bits to be interpreted directly as they're
 decoded.

ec_byte_read() ec_byte_read_from_end() had different return types.
ec_dec_bits() was storing its return value as int instead of
 ec_uint32, which will break if int is only 16 bits.

Also updates the TI dsplib macros to use the same EC_CLZ mechanism
 as everything else.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

For our current usage, this doesn't matter, but is more consistent
 with the rest of the API.
We may want to reduce this to an unsigned char[], but I'd rather
 coordinate that optimization with SILK's planned reduction to
 8-bit CDFs, as we may be able to use the same code.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

This ensures that the various alternative routines in the entropy
 encoder and decoder (e.g., ec_{enc|dec}_bit_logp()) really are
 just specialized optimizations of the same general ec_encode()
 and ec_decode() routines.
This is done by randomly picking one to encode with for each symbol,
 and randomly picking a different one to decode with.

Timothy B. Terriberry authored 14 years ago and Jean-Marc Valin committed 14 years ago

Introduced by 30df6cf3.
This should have only affected the output in the case where the last
 few extra bits caused us to bust, and wouldn't have prevented us
 from detecting the error.

This simplifies a good bit of the error handling, and should make it
 impossible to overrun the buffer in the encoder or decoder, while
 still allowing tell() to operate correctly after a bust.
The encoder now tries to keep the range coder data intact after a
 bust instead of corrupting it with extra bits data, though this is
 not a guarantee (too many extra bits may have already been flushed).
It also now correctly reports errors when the bust occurs merging the
 last byte of range coder and extra bits.

A number of abstraction barrier violations were cleaned up, as well.
This patch also includes a number of minor performance improvements:
 ec_{enc|dec}_bits() in particular should be much faster.

Finally, tf_select was changed to be coded with the range coder
 rather than extra bits, so that it is at the front of the packet
 (for unequal error protection robustness).

This means we're "time-ordered" in all cases except when increasing
the time resolution on frames that already use short blocks.
There's no reordering when increasing the frequency resolution
on short blocks.

Dynalloc becomes 2x more likely every time we use it, until it
reaches a probability of 1/4. Allocation increments now have
a floor of 1/8 bit/sample and a ceiling of 1 bit/sample.