While `x + sin(x)^2` may be monotonic itself, it only takes a simple linear combination of two neurons like `x+sin(x)^2 - (x/2 + sin(x/2)^2)` before you have a completely crazy loss landscape. I have a feeling this is why such activation functions haven't become standard.