I guess it's a bad idea to forward inputs directly

Avoid relying on the order of the layers

Remove the metric because it wasn't too useful and it's buggy in
Keras 2.2.4.

Thanks to changeforan on Github