> They only real negative in my view is the startup time which is noticeably worse than vim for me.

Check that this is not e.g. due to Python host detection/setup - you might have plugins that trigger both py2/py3 hosts to load for example. (hosts can be configured explicitly to skip it) Also check `nvim --startuptime /tmp/nvimlog`.