Now I'm learning Japanese just for the sake of being able to read, write and speak Japanese. Wnet all the way to Tokyo in December for five mornings' worth of Japanese lessons (and some tourism stuff around it); just for the pleasure of learning something of interest.
As you say, a fancy way to waste time. I understand some people do other things to the same ends; movies, television, music, socialising, drugs and alcohol, dancing, art, reading, writing, building things, making things, running, climbing, swimming, on and on and on. Oh, the range of things we humans do as fancy wastes of time.
But you can ask that question about anything, really. What's the point of being stronger, if you can't just go and take other people's stuff? What's the point of being faster, when the first place prize is the same as the last?