If you reject take backs they you aren't playing the most competitive version of your opponents.
Sure, in the moment, I'm not playing against my opponents best choices. But without take backs my opponent is developing the skill of thinking before acting, which makes him a stronger opponent overall.
We only allow take backs for:
1. Newbies. They don't know the rules, so we allow take backs during their learning phase, and also help them understand why the thing they did was non-optimal. This lets us play the game while also helping newcomers learn the rules and general strategies quickly.
2. Misplay. If another player does something not allowed in the game (e.g. plays out of turn) which changes a second players understanding of the current state of the game, we let the second player take back their move if they made it before we discovered the flub.
But chess also has its version no take back a in touch-move requirements. Part of your"best" self is making good decisions the first time under pressure. Or they I'm not playing you, but a small group of you (often with other played input).
Competition involves the stress of making decisions that you cannot take back.
You make a mistake, you learn from it and play better next time. You don't get to take it back. Next time don't do it. If you could always take something back where is the urgency is making good decisions, playing better than your previous self or someone else.
And absolutely not, if you cannot remember what has already been played, well too bad. That's part of the game, especially cards. That's half the point of trick taking games. "are all the trumps out?" Besides the one in office, no way I'm telling you that, and if you are playing at my table, don't answer that for somebody else. You might as well keep track on paper at that point.
Competition is not a dirty word. It isn't anathema to enjoyment. You play to win. It is part of the fun. Competition makes me sharp, I concentrate more, I get that little adrenaline rus when I win. And when I lose... We don't talk and such improbable things.
All games are blood sports.