I’ve played piano for the last 20 years, starting off initially with a teacher and going through the usual sheet music pieces. I used to think as you do for the longest time - “The important thing is to have fun!”

I still believe that, but I find that I learn new techniques every time I read, learn and play an existing piece - this makes my improvisation and jam sessions all that much better! So would highly recommend learning to read sheet music.