Enhancements the fixed-point approximations of non-linear functions.
Accuracy for rsqrt, rcp, cos, and log2 is now at the level of truncation error for the current output resolution of these functions. sqrt and exp2 still have non-trivial algebraic error, but this cannot be reduced much further using the current method without additional computation. Also updates the fast float approximations for log2 and exp2 with coefficients that give slightly lower maximum relative error. Patch modified by Jean-Marc Valin to leave the cos approximation as is and leave the check for x<-15 in exp2 as is.
Loading
Please register or sign in to comment