Skip to content
Snippets Groups Projects
Commit dc8a6eee authored by Timothy B. Terriberry's avatar Timothy B. Terriberry Committed by Jean-Marc Valin
Browse files

Update the log2 approximation to accomodate the increased bit precision from... 

Update the log2 approximation to accomodate the increased bit precision from bd0610d2. This requires another term in the power series to reduce the error to the level of truncation (peak absolute error 0.621785). Also refactor the rounding bias term so that further changes to DB_SHIFT will leave the result unbiased.
parent 3a4a463f
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 4 additions and 5 deletions
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment