Jean-Marc Valin authored 3 years ago and Jean-Marc Valin committed 3 years ago

Down to 20 features

Only on the C side, no sparse GRU B training yet

Weights are the same, but they are dumped to C differently.

Adapted from PR: https://github.com/mozilla/LPCNet/pull/134
by zhuxiaoxu <zhuxiaoxu@ainirobot.com>
but had to be reworked due to previous weight quantization changes.

Makes loading faster

AVX without AVX2 should now work again too.

Avoids having to compute a sigmoid

It no longer overwrites its input vector

The 4* is now stored in the table to avoid computing it in the loop

Saves on the MDense/softmax computation since we only need to compute
8 values instead of 256.

Using rational function approximation for tanh() and sigmoid.

CuDNNGRU and GRU don't use the same weight format

This isn't necessary since valid exponents can't flip the sign bit

See https://github.com/mozilla/LPCNet/commit/bffdcee95#commitcomment-46372726

When implementing using SSSE3 or AVX2, our dot products can saturate
if two adjacent weights sum to more than 127.

Not sure why CuDNNGRU doesn't get used by default, but we need
to explicitly use it to get things to run fast.