When I removed an old commited and then removed node_modules folder, I had to do a git filter-branch and a force push; it's doable for your own projects if you don't object to changing the history a bit.