I think my psychedelic experiences mirror yours quite well, and it has led to me reaching much the same conclusion, though you've put it into words much better than I've ever been able to.

Have you ever experienced your sense of time becoming immaterial? I've had one such trip where I genuinely believed that time stopped, and I was present in some eternal plane, though my ego no longer was present. I can definitely see how such an experience could lead someone down a path of panpsychism.