Replace log2_frac in the delta offset calculation.
Adds a new bitexact_log2tan() function which is much simpler, and more accurate. The new approximation has an RMS error of 0.0038 bits from the correctly rounded result over the range of inputs we use, compared to an RMS error of 0.013 for the old log2_frac() method. The actual computation of delta is also changed to use FRAC_MUL16, since this allows us to keep the full accuracy of the new method while avoiding 16-bit overflow. The old delta computation actually could overflow 16 bits: it needed 8 for the log2_frac() result, 1 for the sign of the difference, and 8 more for N.
Loading
Please register or sign in to comment