It's not just children's music. Most songs you find today, if you listen closely enough, the words follow the beat. I.e. every once in a while, a word starts on the "one" beat.
And we take it for granted. You'll very quickly realize something is "off" if that's not the case. That's usually what you find in "artsy", "look at us we're weird and different" type of music. Annoying most times, and incredibly difficult to get right.
As it happens I took classical piano as a teenager, but always had trouble with the rhythm.
When getting a medical exam for scuba there was a suggestion that I had an arhythmic heartbeat and I've always wondered if that could relate - perhaps my brain just can't do rhythm in any way at all ...
On the anecdotal side, I've played music with a lot of people over the years, and have long suspected there is some sort of inborn talent spectrum for both rhythm and pitch discernment (somewhat separately from each other, too). Practicing a ton will make a difference, but it definitely appears that certain skills come much more easily for some people than others. Good drummers are actually pretty hard to find in my experience, and they tend to be in high demand.
When taking lessons in a music school I also had to follow separate classes of music theory which I hated, so I wanted to quit. But my teacher desperately wanted me to continue and allowed me to follow his classes "unofficially". Then I started playing in bands and the other members were always really impressed. Again I didn't understand why.
I'm at best mediocre at everything else I do: other musical instruments, being a developer, these kind of logical reasoning intelligence & maths tests.
Don't really excel at anything at all but I always felt that playing drums was some kind of built in talent I had, so I believe there's a kind of person that's born with a real talent for rhythm.
When I learned how to play in the first place, particularly with jazzy bands and all that, it honestly felt like the mental rigors of studying, since I was trying to incorporate rhythms, as well as breaking them, all while giving a steady heartbeat to the band playing with me. Learning different techniques, handling, sticking, getting to the point where hitting the drum head at the precise location for the precise sound needed was a lot of multitasking to keep in mind, while holding steady notes on a snare or a hi-hat.
And the point was to practice all of this until it became second nature, and you didn't have to keep any of it in mind as you played.
So it'd be more fair to say that drumming kinda rewrites the way you think and redirects certain aspects of your mental faculties.
But we all get your point ;)