It's really just a personal intuition. I don't have any good argument or explanation to offer...

For what it's worth, I do feel that panpsychism was my "natural" belief when I was a kid, until the adults told me it was crazy to believe in that. I still don't think it's crazy, and there are some people who I respect a lot who believe in panpsychism.