... I am guilty of downvoting and moving on a couple of times.
Guilty, guilty, guilty! :-)
Seriously, I think all of us have done it. But for most it's an occasional thing and we wouldn't miss it if downvoting went away, since most of the time we either write an opposing view or upvote an existing one.
Up == agreement
Down == disagreement
Flag == violates community standards
I generally try to add a thoughtful response to anything I downvote, but I am guilty of downvoting and moving on a couple of times.
I think requiring a response to downvote would be better than removing downvoting entirely.