When you're 20-30 something and a nobody in your field, but a privileged mind, you keep your eyes open to all posibilities. You are hardworking and then results come.
Then you gain a name and have all eyes on you and your mental processes change. Suddenly with your opinions, you've got a reputation to defend. You're aware of those supporting you and those who are not. You're reasoning gets blurred by your ego and personality. Once you publicly speak an opinion on a topic for enough time, you cannot simply go back without looking like a moron.
There are several instances of the most brilliant minds following this pattern. Geniuses giving their backs to the scientific method, to all evidence, and looking like amateurs.
Comes to mind Einstein, spending his last years trying to debunk quantum physics against all evidence, Linus Pauling and his C-Vitamin craze or Nikola Tesla and his every time crazier projects with no practical viability. I guess there might be more.
I suspect the ego/arrogance thing works both ways. When you're younger you're often more willing to show the finger to existing paradigms and are perhaps arrogant enough to think you can "revolutionize physics." Naturally, most people can't do that but some make history.
(Also, I'm talking out of my ass and should probably add some history of science to my reading stack).
I’m not sure this had to do with ego. He just seems to have enjoyed building crazy stuff.
It likely begins to decline even prior to that, as our ability to measure it is very rough and imprecise, which seems to align well with ages when physics/math geniuses peak, ~30s.
I'm only going to give a small part of my thoughts, there's more, but let's keep a narrow focus for the forum format.
I'm less and less willing to submerge myself deep into fields far removed from daily life. I can still do it, and actually easier than ever before. I just don't want to. I feel it's not because I'm dumber, quite he opposite. It's because I understand and see sooooo much more than twenty years ago. I just don't see the point as much. I had a much more narrow view in my youth, and that allowed me to concentrate on narrow and deep subjects for long periods of time. I don't want to go into the reasons because that would really explode the text volume and mix subjective and objective things that would in turn require quite some effort in untangling. I also care quite a bit less about "progress" because I lost a lot of idealism. In that sense, understanding more of the world might be counterproductive for productivity...
I think I can claim I'm a lot more capable with my brain than two decades ago. I tested it by taking numerous edX and Coursera courses, quite a few of the very challenging, and I was invited to become a C[ommunity]TA several times in courses way outside my own field, including math stuff that I had difficulties with in the past. My job is not standard stuff either.
I don't think chess players peak at 35 because they're losing their touch. I think they develop other interests; the ones who aren't going to be professionals realize it and downshift, and the truly elite ones achieve what they set out to do and move on to things they see as more interesting.
This is probably why the corporate thugs are so ageist. They know it's easy to pit people in their 20s against each other. Once people are older and have more perspective, it's hard to make them compete over scraps.
However, with older age comes ability to select where to focus better and put things in perspective. The skills gained with aging don't really help with chess, which reduces most of the time to raw calculation and focus.
There's a tendency for exceptions to drive the narrative, which leads to a sense of mathematicians peaking earlier than they actually do. Mathematicans do indeed peak "young", relative to other fields, but most often in their early to mid 40s. Of course, at that point they're ineligible for the Fields Medal, but that doesn't mean they're no longer productive.
Writers peak around 55, but the drop-off can be steep, not because they lose the ability to write, but because revision and publishing can themselves be brutal processes. Revision is brutal because of perfectionism combined with the inability to hold a whole book's worth of context in your head. You can't look at the whole thing in one sitting, as with a painting. You have to edit at specific levels (themes and story arcs; then subplots and characters; then scene structure and transitions; then paragraphing and flow; then line edits; then copy edits; then grammar) but they sometimes compete for your attention. Writing work that's good enough to publish and sell is easy; writing work of literary quality is much harder. Publishing... well that changes every generation. These days, self-publishing is the right play for 99% of authors, but that brings its own challenges.
With age, the thing that gets hard the quickest is long-form context switching. Very few people can (or want to, or should) dedicate their lives wholesale to one pursuit. Once you start to get recognition and are invited to the parties, it gets harder to focus on writing. You spend a lot of time around non-serious players, and you become one. Lots of people turn stupid once they get popular and spend too much time with the fun people to have real thoughts anymore, and that's certainly not limited to the old. The difference is that the young can retool themselves more quickly. A 27-year-old can bounce back; he can be on cocaine-fueled benders in January, courting the nonserious players necessary to get the "tastemakers" to support one's career, and be back in his writing room by March or April. When you're older, this gets a lot harder. Older people are great when they establish momentum (leading me to question the "old people are less creative" hypothesis) but do not bounce back as well from periods of distraction or excessive comfort.
I recently went back to university, and they tend to have exams that favor the young: lots of memorization and much more questions than there is time to answer. So I probably would have done academically better with my younger brain, however, I feel like I understand the material and the context much better than I would have before.
This type of thinking does not help. There is no magic, there is no enchantment, there is only vision - the ability to see, to imagine, what others fail to imagine. Then it is an issue of communication, to tune others so they can imagine the new entity too. I believe this is a communications issue, and those individuals with intellectual gifts have an inner voice that communicates in non-complex basic ideas the gifted see transparently inside complexity.
Case in point: when first learning about normalized numbers, a friend smarter than I in the same class remarked "Oh! they are talking about my 'minis'!" - my friend had figured out, realized the value of normalized numbers when learning basic multiplication in elementary, assumed the concept was obvious and incorporated it everywhere he used multiplication - to the ire of his elementary and high school math teachers. I lost track of him, but I last was aware of him being a cross disciplinary professor at Carnegie Mellon University.
About painters and other visual artists, it's true that many were still working and producing in their final years. OTOH it was often the case that in old age they worked in relative obscurity as younger artists with "modern" styles had become the ones sought after and celebrated.
I've come to understand that the merit of most contemporary visual artwork won't be decided until 50 or 100 years down the road. In his time Rembrandt was known mainly for his magnificent etchings, printmaking was his main business. Now we celebrate his paintings as much or more, but we have the luxury of centuries to have honed our esthetic metrics.
My hunch is that creativity among physicists is the product of a special kind of insight, a moment when a number of seemingly disparate (and possibly mundane) phenomena suddenly align within a brilliant mind. Maybe this is a pretty common experience, but in most instances the insight isn't about anything really important to the rest of the world. Mostly it's something that happens early in life, young adult years most likely. Careers are about working out the details and reducing it to words (and equations) others can understand, a very hard task to accomplish.
While much has been written about creativity we really know very little about what magical things make it happen in those rare and enchanted brains.