A newborn foal can stand within 55 minutes of being born and can walk or run within 90 minutes. That is crazy fast training speed. Is it training initialization or is it a form of transfer learning?

Most human babies take 9-18 months to walk, and this is commonly attributed to the fact that human babies have immature brains that cannot mature due to birth canal constraints, which is constrained by pelvis size which is constrained by the need to walk upright.

I have wondered about how true this is. Maybe our brains are just so much more powerful, or capable of much deeper understanding, that we require a much lower initial learning rate.

The lottery ticket could explain why some children learn to walk at 6 months, while others are closer to 24 months, with no different in intelligence or motor skills in later life.

Edit: As some people below have correctly pointed out, human babies could not walk within an hour for physiological reasons, but nor do they exhibit the basic motor skills, spatial reasoning or image processing required for walking. Many human babies even struggle to feed for the first 24-48 hours or longer.

Meanwhile baboon babies can hold onto their mother from birth while the mother climbs trees.

Interestingly, the EGG hypothesis postulates that the constraining factor is not pelvis size, but rather that neonates consume so much energy that we give birth just before a women's metabolic limits would otherwise be overrun (leading to both maternal and infant death). Based on this, the increased energy demands of our larger brains mean that we have to sacrifice neonatal motor skills or drastically improve the metabolic efficiency of adult women.

https://www.pnas.org/content/109/38/15212

> In humans, maximum sustained metabolic rate is thought to be 2.0–2.5× basal metabolic rate (BMR).

> By 9 mo, metabolic demands of the fetus threaten to push maternal energy requirements beyond 2.1× BMR. Extending gestation by even one month would likely require metabolic investment beyond the mother’s capacity. Instead, the mother delivers and the neonate’s growth rate slows relative to its fetal growth rate, keeping both the offspring’s and the mother’s energy requirements in check.

Worth noting how massive the growth rate is I think: Even after birth, the baby gains weight faster than it ever will again in its life. In absolute terms! In relative terms it's even more insane.

A 12-year-old boy going through puberty will go from 90lb to 100lb over a year, an increase of 10lb or ~10%.

The average baby boy will be born at 7.5lb and weigh 20lb at one year old, an increase of 12.5lb or ~270%. Nearly triple the size!

The growth might seem impressive when you do percentages of a very small number, but it's still a very small number overall.

If not--I am very lightweight right now, and I'm trying to put on body mass by eating more and exercising. While my volume will increase, my main growth is in density. My growth is not in fat, but actual muscle mass. Is this growth? If so, I have gone from 125lb to 140lb in ~2 months, which in absolute terms is obviously more than the baby (my goal is 160lb by the end of the year).

A few years ago, I went from ~130lb to 165 lb in three months--blame the free food at FAANG cafes during an internship. But this might be closer to what you were saying earlier--putting on fat.

Edit. I guess in the original statement saying that a person will never gain weight at this rate again there should be the explicit assumption that we're talking about natural growth, not growth due to external factors (ie more eating--intentional or not). In that case, that statement would be true.

But in that sense, of course a baby will have more growth than an adult--isn't it reverse exponential?

Average height at age 12: 59". At age 13: 62". 3" of growth.

At birth: 19". At one year: 29". 11" of growth.

Try matching that just by eating more.

(obviously, yes, it's easier for a small thing to grow than a large thing. But it's still crazy fast. Imagine being 50% taller in a year)

This seems weird. Fully breastfed babies require even more calories from the mother. (They are bigger than unborn babies, and they lose more heat.)

In contrast, by the third trimester of pregnancy, your total blood volume has increased by approximately 50% (3-4 pounds of additional blood), not to mention an extra 2-3 pounds of retained fluid, 2 pounds of amniotic fluid, and 1.5 pounds of placenta. All told, this is another ~10 pounds on top of the baby. Additionally, the mother / placenta is performing all respiratory and blood filtration functions on behalf of the fetus (the kidneys don't really do much prior to birth other than produce urine).

> https://www.mayoclinic.org/healthy-lifestyle/pregnancy-week-...

Personally, while breastfeeding and newborn care was exhausting, it felt more akin to training for an athletic event vs having another entity symbiotically tied to my system like pregnancy.

I was mostly just remarking on the energy-demand hypothesis advanced in the comment above.

My first daughter is just in the process of weaning at the moment.

Growth rate slows after birth, so that would account for some of that. Mother’s also would not need to carry around the weight of the baby, and would be more able to provide for the child.

Though it's not 24/7.

Since babies traditionally breast feed and thus survive solely on the mother’s energy after birth, this explanation doesn’t seem to hold water.

I kind of doubt that, because if baby is after term, the risk is supposed to be childbirth itself being too dangerous. And they induce birth if the baby is too big too.

Large baby is considered a risk on itself.

https://www.stanfordchildrens.org/en/topic/default?id=newbor...

Perhaps the most relevant to this discussion is the "stepping" reflex. It isn't really stepping but if placed in a standing-like position babies will often raise one foot, put it down, then the other. The most interesting part to me is that this reflex stops by two months. Makes me wonder if it's an evolutionary leftover from our days as monkeys or something like that. It's all so fascinating.

I read that both failure to demonstrate these reflexes while newborn, as well as continued existence past a few months old, are signs that something might not be entirely right with the baby.

Have you ever seen a new-born baby's legs?

They aren't even remotely strong enough to walk, no matter what the brain tells them. It takes those months to develop leg muscles. New-borns don't have them.

It's absolutely not restricted by the brain size.

The brain size needs a huge head, so in a sense it is.

All the factors are interwined. The growth of the brain seems to have been a very fast evolution, The pelvis had no time to adapt, so giving birth early was the final solution.

With that in mind, I don't find fair to shame the pelvis, or the brain. The weakest link is the legs, but it's not their fault either since they had no time to mature.

Give one of those cocky foals a head of proportional weight and allow them to use only two limbs and see who's stumbling now!

Edit: newborns can't even balance their heads on their necks, they're extremely fragile.

The balance comes much later than the strength.

If you start occasionally holding your baby up by the thighs when they are 4–5 months old (their back/abdomen muscles at that point are typically plenty strong), after about a month they will have the balance to hold their torso upright when sitting on an adult’s shoulders, for at least a while. By age 8–9 months they will be very stable and good at balancing for long stretches of time. (I recommend any new parent try this, not only for the baby’s sake but because carrying a baby on shoulders is dramatically less tiring for an adult than carrying a baby in arms, and less hassle than a stroller or baby carrier.)

People generally hold their babies from the top (e.g. with hands in armpits), in a way that they are passively stable. If you want to train someone’s balance, you want to force them to actively stabilize themselves from a default-unstable position at least a bit every day. Babies have very short torsos and small heads compared to older kids, with the effect that their torsos have a much smaller moment of inertia about the waist: it takes several times more muscle strength for a 3-year-old to lift their torso up.

Do you mean baby sits on my shoulder with his legs around my neck, I am holding on to his thighs, and he can hold on to my head with arms? We both face forward in same direction.

Then after a month or two of practice (a few minutes per day, nothing too serious here) you put your baby on your shoulders, holding his ankles, and he will be able to hold his torso up with his abdominal muscles for at least a few minutes at a time. Practice this for a few more weeks and you will be able to walk around town with the baby on your shoulders.

If you get a few 3-year-olds and e.g. get one to practice kicking a ball for 10 minutes per day, another to practice riding a 2-wheeled scooter, another to practice hanging from monkey bars, and another to practice swimming, after a few months there will be a wide gulf between their abilities at those skills.

Anyway, hard to conclude, just think your logic is missing a little.

You have this a bit backwards- yes brain size at birth is limited by pelvis size, but this means that humans are born prematurely compared to other mammals. The entire body is basically premature and underdeveloped, not just the head/brain. If anything the brain is relatively over-developed, as it's size is the constraining factor in gestation length.

If I remember correctly, 18-24 months is the estimated gestation time humans would have if not restricted by pelvis size (which ironically is itself a result of walking upright), and this fits better with the idea of being able to walk closer to birth.

There's a chapter in "Born to Run" that ties human physiology and endurance and evolution and brain development together in a way that's so elegant, it almost has to be true. It covers the steps that lead to the need for a bigger brain and walking upright as a means for persistence hunting. Highly recommended reading, especially that chapter.

I doubt it's randomly initialized. The brain has centers for motion which might be honed in at birth for these animals.

Let's not ignore genetics, which pre-populate weights and subnetworks in the form of motion-control centers in the brain. Once you consider the gene-brain interaction over the course of evolution as part of your training, the huge data required for modern ML starts to make more sense.

I strongly doubt that human babies are physiologically capable of walking before 6-9 months regardless of their level of brain development. The entire baby is born many months "early" compared with other mammals due to the limitations of the human pelvis.

[1] https://www.scarymommy.com/complete-shock-early-walking-baby...

Our eldest wasn't interested in crawling really and went almost straight to walking but not until about 14 months IIRC (it was > 10 years back).

We did baby sign (a sort of simplified BSL; would recommend) and some people suggested it would retard speech - same child modified a sign before 12 months and so could tell us he needed a poo/wee and we'd sit him on the potty to do it all before he was able to talk - saved me changing a lot of stinky nappies! Longitudinally, our kids used baby sign and did/are doing well at school and have all been ahead with language skills (but that's not accounting for confounding variables).

It's curious to see infant developmental stages becoming more and more to be compared to AI/robot development.

The way I understand it is that is not so much required as not selected for. We could certainly be able to walk at one day old if evolutionary pressure made it necessary, but it prioritized other aspects in our cognitive development to survive so we don't. It's not as if we say "oh, horses can walk early and therefore are incapable of rational thought" we evaluate them on their own merits.

Is it? If we ignore compute requirements, how long would a good algorithm take to learn something like balancing a quadrupedal robot, based on real-time feedback to its outputs? It is a simpler problem, but drone flight software based on learning, can re-learn how to fly a quadcopter after something like losing a propeller and a sudden shift in weight distribution, in a few seconds.

A robot could be built that learns how to adapt to something like a limb being lost or added or half its weight shifted to the other end. It could probably learn an approximate optimal to move under those conditions in just seconds, as well. Though, I suspect that foal might actually be nearly as adaptable. Adult horses, or humans... not so much. We have many overlapping models of how to move, probably. I would take only seconds to adapt to half my weight being added on my shoulders, in the same ballpark as our best robotic systems. But if my left leg grew four inches it would take me a lot longer than the robot to learn and internalize the best way to move again. I could barely keep up with that when it was just a few inches a year when I was a kid.

Seems easy but why don't we have self driving cars by now?

Walking it not just a mechanical task. Your quadcopter is not learning to interpret raw electrical signals from an IMS and raw electrical signals from a CMOS sensor. The training is also probably done in a simulator, where if it breaks, it is reset and training continues.

A horse is learning to interpret inner ear signals and signals from the optical nerve AND the using those to build a spatial map of the environment. Only then it is actually trying to walk.

A 2 hour old baby horse can not only run on not-flat terrain but is spatially and environmentally aware. It has taken years for Boston Dynamics to develop a platform that is equivalent to a pack horse and even then probably can't respond independently to never-seen-before threats or challenges.

Trying to build a neural net or set of neural nets that can do all of those jobs is a daunting task and no one has the time to train the artificial neural network for 15 years. that's why we don't have safe self driving cares yet.

as for the horse, well 50 millions of years of evolution on constantly refined genetic algorithm is hard to beat and that was on top of the evolution of the eohippus as a starting point

Firstly, the drone software is not re-learning anything; when a baby walks unaided for the first time and you place a rattle in their hands, do you really refer to that as "re-learning walking"?

Secondly, forget about computational requirements and just look at how many iterations have to be run to reach success; mammals (and humans specifically) can "learn" to recognise an entire class of something just from one or two images of that something. Learning something else doesn't "forget" the old thing. A NN trained to recognise faces in images can't, to me knowledge, be trained to also balance a bi-pedal robot, or ride a bicycle.

IOW, what I see currently is that NNs are very one-dimensional - the net that becomes optimised for one task becomes unoptimised for that task if you subsequently add new tasks.

Or maybe having 4 legs (as a foal) gives more natural balance than a human or maybe even a cat or dog.

In the same way technically you only need one leg to walk (hop) around but that's much more difficult than having two.

However, another commenter brought up the fact that human muscles are somewhat underdeveloped after birth. It might make sense that having 4 legs is easier both in regards to balance, as well as because the load per leg decreases to make standing on them easier.

That is probably not the only reason, but probably a contributing factor.

Probably neither. How does the human baby "learn" to breath almost immediately after birth? I suspect the answer here is also the same.

https://www.stanfordchildrens.org/en/topic/default?id=newbor...

> Stepping reflex > > This reflex is also called the walking or dance reflex because a baby appears to take steps or dance when held upright with his or her feet touching a solid surface. This reflex lasts about 2 months.

Maybe not enough to qualify as "exhibit the basic motor skills"

It isn't. Human babies prioritize differently. We learn to understand language and to communicate before we learn to walk.

(In fact, for many babies learning to walk is a conscious process. We learn to learn before we learn to walk.)

Isn't it just a matter of priorities? Many birds spend a similar proportion of their life unable to fly, but they're safe in a nest. If you're a foal on the savanna, you need to get up.

Hence why the lottery ticket network (being a smaller version of the same initial network) can achieve similar accuracy (but not 'better') in much shorter time - because you took away all the confounding stuff. That kind of 'counterfactual' doesn't really translate to a biological domain, right?

Also - the baby brain is not really a 'pruned', smaller version of the adult brain, right? Neither is the fetus brain compared to the baby brain (which is more to the point of the article). So the analogy breaks down there too

I guess what the article describes is a bit like "pre-training" a full network with 'synthetic' experiences that quickly tune it for later. In that sense, I think its like these "dreams" are a 'distilled dataset' [1]. The question then is: how is this dataset being passed on from mother/father to child?

[1] https://arxiv.org/abs/1811.10959

Abstract:

> Model distillation aims to distill the knowledge of a complex model into a simpler one. In this paper, we consider an alternative formulation called dataset distillation: we keep the model fixed and instead attempt to distill the knowledge from a large training dataset into a small one. The idea is to synthesize a small number of data points that do not need to come from the correct data distribution, but will, when given to the learning algorithm as training data, approximate the model trained on the original data. For example, we show that it is possible to compress 60,000 MNIST training images into just 10 synthetic distilled images (one per class) and achieve close to original performance with only a few gradient descent steps, given a fixed network initialization. We evaluate our method in various initialization settings and with different learning objectives. Experiments on multiple datasets show the advantage of our approach compared to alternative methods.

I think it is the other way around -- adult brain is a physically pruned version of baby brain. We start with lots of connections from everywhere to wherever and then lose a lot of them as we learn, while strengthening useful ones.

https://www.smithsonianmag.com/smart-news/dragonfly-undertak...

Insects that have large migration patterns - all of the individuals die before coming back to the initial spot; only their descendants continue the cycle. How?!

In addition think about how bees can tell harvest locations to each other by dancing: https://en.wikipedia.org/wiki/Waggle_dance This dance is probably also something those bees know before being born.

Or... has anyone tries training a network for determining the initialization state of another network? And now I'm wondering if those good starting initialization values only work well for a specific task or if they seem to span all or a subset of tasks, and maybe there's some inherent quality in how the values relate that we can tease out...

  In the days when Sussman was a novice Minsky once came to him as he sat
  hacking at the PDP-6.  "What are you doing?", asked Minsky.  "I am
  training a randomly wired neural net to play Tic-Tac-Toe."  "Why is the
  net wired randomly?", asked Minsky.  "I do not want it to have any
  preconceptions of how to play." Minsky shut his eyes.  "Why do you
  close your eyes?", Sussman asked his teacher.  "So the room will be
  empty."  At that moment, Sussman was enlightened.

See "Weight Agnostic Neural Networks" (https://weightagnostic.github.io/):

> Inspired by precocial species in biology, we set out to search for neural net architectures that can already (sort of) perform various tasks even when they use random weight values.