And what I found when I started doing that was that suddenly I was a lot less hesitant to let myself become dependent on time-saving tools because I knew they'd be available anywhere with ease.
I can't imagine suffering through a default shell environment or a default editor config any more.
But committing them is not just about remote machines, but about having a clean record of what is needed to set up any new machine I acquire as well.
It's not just about having my editor available, but being able to trivially easily pull in the basic tools to do what I want, and that is especially important during emergencies, but it's also about convenience in other situations.
It sounds like you're lucky enough to deal with very simple environments. That's great. I'd still argue the cost in time of committing your config is so low that it'd be worth it. It takes pretty much one reinstall before it's paid for itself, or one instance of being somewhere without your usual environment.
Or one instance of doing something silly to a config and forgetting what you changed.
The nice thing about it is that it can start extremely light-weight - just literally a "git init" and .gitignore everything, and then gradually force add and commit.
There's a few tweaks I miss when working in a remote shell but none are essential.