Children are not small adults, and babies are not small children either. It is not like your present self going to a tennis class on the weekends and leaving it aside the rest of the week. Every stimulus during the first few days/weeks of life outside the womb has a disproportionate effects on the neural development of the individual.
Using a computer engineering analogy, babies are "pre-programmed" by millions of years of evolution to "bootstrap" their brains through "machine learning". Mess with the "trainning set" and you will cripple the result for a lifetime.
This is the kind of experiment (on mammals) that can get PETA all over you. Just considering its hypothetical use on humans is pretty much the stuff 007!Mad-scientists are made off.
Hopefully it remains hypothetical and does not become a mad-scientist methodology for using babies to bootstrap a neural network that teaches robots how to walk.
I was thinking more in the lack of practical purpose of the endeavor itself[1]. Something that potentially causes a bunch of collateral damage for no clear purpose. Human babies have been learning to walk on their own for about a million years unaided, just like every walking animal before them.
[1] Lampshaded here (kind of): https://www.youtube.com/watch?v=qnMaroAlfvI