Skip to content
Snippets Groups Projects
Commit a9ffc14a authored by Timothy B. Terriberry's avatar Timothy B. Terriberry Committed by Jean-Marc Valin
Browse files

Enhancements the fixed-point approximations of non-linear functions.

Accuracy for rsqrt, rcp, cos, and log2 is now at the level of truncation error
 for the current output resolution of these functions.
sqrt and exp2 still have non-trivial algebraic error, but this cannot be
 reduced much further using the current method without additional computation.
Also updates the fast float approximations for log2 and exp2 with coefficients
 that give slightly lower maximum relative error.

Patch modified by Jean-Marc Valin to leave the cos approximation as is and
leave the check for x<-15 in exp2 as is.
parent ab4dcc5c
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment