I was not good enough to play an unknown variant and come out better, as you were!
So over time I drifted to variants like bughouse or speed chess variants, to play for fun and not feel like it was studying or being constrained by the standard moves.
And then boardgames took over; for me personally I need to have a mix of luck and skill in a game to enjoy it.
I do this trick a fair amount in Go, and actually have accumulated some "standard" non-standard openings that I play when the oppurtunity arrises. Because I play them fairly often, I actually have more experience with them then my opponent, so end up with the "memorization" advantage.
Also, in my experience, players will notice the mistake and then try a bit too hard to take advantage, instead of being satisfied with getting the "normal" result.
Nb however that most tournaments and such ban loads of bidding strategies with all kinds of special rules, so you can't do this just anywhere.
I broke through a plateau by studying openings more than memorizing them. Slowly working through the early possibilities to separate what works from what doesn't helped to reinforce basic principles.
So like hammering away at 30+ identical problems in high school math class, with some tedium it really did help me internalize the principles. I did not continue to memorize openings and common lines to any great depth. I remain a meager, but much more contented, chess student.
There is a certain skill level, which I want to say is somewhere around 1400 to 1500 but it has been over a decade so don't quote me on that, where most the players at that level seem to focus on memorizing openings. It ended up being what separates them from the players a one to two hundred points lower. This leaves them vulnerable to a bit of a hack, as using a non-standard opening can completely remove any usefulness of their memorized openings. It almost disorients them and makes it much more likely for them to make a mistake that can then overcome the disadvantage of such a non-standard opening. I loved playing in this area as I hated memorizing openings so I just used non-standard openings to not to avoid having to. Eventually I hit the ranking where players were good enough to take advantage of my non-standard opening more than any advantage from disorienting them and I quit playing because the only option to advance would've been to go back and memorize openings.