I've done the same (with autofill switched off in my password manager). Makes a real difference.

I should delete the app from my phone too. Or at least switch off all notifications and move it into a folder far away from the home screen.