bebimbop

That doesn't follow. Predictability is your brain reacting to the lack of novelty. What an LLM produces is a chain of words following some kind of conditional distribution, whose output could vary significantly from your understanding of novelty depending on its training and priors.

With a single, generalized prompt ("produce something unpredictable"), ChatGPT responded with:

> 00:— / ANOMALOUS RESIDUE > Vekh. > Ìssara. > Nnnn—tq—èl. > > A temporary thickness enters the meat of the signal. > > Not rain. Not a substance with permission to fall. > > UL / 9 / ferrous / ∂ > The measurements are eating each other. > > Someone has left an entire childhood in the wrong unit of pressure.

That is not predictable to me, at least. It could be boring, but I think that is mainly a product of me knowing where the text is from. LLM-generated text is inherently boring, to me, but that has nothing to do with predictability.

> everyone agrees that it's the result that counts.

I don't know if I agree with that. That presumes that the entire point of mathematics is to produce results. I don't believe that at all.

I believe that the entire point of mathematics is to produce human understanding and mental models that are useful or beautiful to humans. (I also believe there's a mysteriously deep connection between those two notions, but that's a different discussion.)

At any rate, I think that the old process, where humans had to cut the paths themselves before the clearing emerged, is one that led to understanding first, then a distilled result. And then the secondary task of distilling the understanding down to a clean proof is the icing on the cake. Some mathematicians are very good at keeping the intuition and understanding clear when presenting proofs. Some are not.

However, very often, the end result and proof is not where the real understanding lies, and that is one of the reasons why mathematics has been such a social endeavor. Often you can't glean the "hidden truths" behind austere proofs before you engage with people who can help you see the picture behind it. Many people who went through Baby Rudin as an undergrad can maybe relate to what I'm saying here. For a very insightful analysis about how intuition and discourse lead the discovery of mathematical truth, I highly recommend Imre Lakatos' amazing book "Proofs and Refutations."

What the models are producing is not what we had before. They are producing the cargo cult equivalent: Equations and words on paper which by means of tracing through the steps might take you from A to B. And after a while, humans might discover the true facts behind why that path from A to B exists, and then understanding emerges. And only then.

But when the process is done through AI, no one (no human) learns how to make discoveries and connections on their own. We are not letting people exercise the cognitive apparatus which is used to gain intuition and understanding. My fear is that after a while, fewer human mathematicians are able to gain the understanding mathematicians before them did.

It's as if someone removed all the barbells from a gym and made automated hydraulic jacks do all the lifting. After a while, people would wonder why our power lifters aren't as strong anymore. "But weights get lifted! That's the entire point, right?!"

Maybe, but cement manufacturing by itself is responsible for a staggering 8% of global co2 emissions. And we really don't have a viable option for replacing cement, and probably won't for a long time. Capturing and removing that co2 has a meaningful impact. Even if CCS were limited to just that one industry.

It's good that all of these benevolent billionaires are bestowing the world with the fruits of their goodness, and make sure that assets, knowledge and access to the market are fairly distributed. Their political involvement is to the benefit of all of mankind, and does thankfully not lead to the centralization of assets or power in one small group.

Oh Brave New World.

Now that all of the trite and boring intellectual work is automated, I can look forward to having it bottle-fed back to me at night. Now get me a gramme in time...

Thank you for putting this so eloquently into words. I feel the exact same way.

The journey is the destination. I don't care how many checkpoints you can breeze through, if you don't grow, have an experience that transforms you through the process of doing it. And if you're just ticking boxes by means of having a machine do it all for you, there was no need initiating the process in the first place.

I am not talking about producing tools that are means to an end here. I am glad we can make millions of socks a week by means of automation. But intellectual endeavors are different.

The point of doing math is not proving theorems. It's to discover the intuition, the underlying phenomena, internalizing them and adding to the human understanding of the problem. Writing out the proof is just the incidental outcome of that process.

If no human learns anything, if we just start producing testaments to a human null process where no person has emerged strengthened from it, we are no better off than we ever were. We are just producing empty and meaningless representations of something which is not there.

I am not sure how convinced I am by that argument. A gun is also a particular kind of tool, and it makes the person at the handle end sovereign, and the person at the pointy-shooty end subjugated.

There are other things that point to Gates being more than average in terms of mental aptitude, like being among the very select group of Harvard undergraduates who took Math 55.

I had no problem topping it up with apple pay while in Japan.

I got one from him a couple of years ago. He even sent me photos while packing and shipping, and a handwritten note along with the bottle. An absolute stand-up guy.

I doesn't seem so much an argument against their intelligence, as an argument against their moral character. We're not at a point where we can get models to act in accordance with what we'd call moral integrity.

I don't know if I'd consider GitLab, Bitbucket or any of the other git repo hosting services to be much worse than github in terms of availability or reliability.