arenz

Most new programming languages come out of academia and are mathematical toys proving some aspect of type theory. They are as flawless as they are useless, more demos to backup a paper than things of independent value. No one attacks them because they make no claims that are not proven with 20 pages of denotational pedantics.

And then other languages come out companies (Apple, Facebook, Google) and are backed by endless money and instant communities of their own internal users. The languages are made by teams of elite engineers and enjoy instant institutional credibility.

It is a hard world for the independent language creators. I know, I've burned a decade on my own language project. So I am pretty forgiving.

I find Alex's wild, "I'm sure a solution exists to this problem and I'll solve it when I have the spare time", philosophy rather refreshing. And the fact that people write about his work and that he has an active community shows there is something to the project. I suspect people in the community have workarounds for the missing bits and treat the unsolved problems as goals.

If Alex had been conservative, never sharing until he was done and never risking a promise he might not keep, would you know his name?

Shit, your title gave me shivers. This is exactly what I feel online. Less and less is real and it is getting harder and harder to tell. There is so much noise, even the real voices are not often heard.

Cool work! It is impressive how well the LLMs do in specialized tasks like this.

I'd be interested in seeing how they compare to old fashioned neural networks terms of power, time, and accuracy.

I think the speed gap between the LLMs and narrower neural networks is similar to the gap between human workers and the machines that replaced us in the industrial revolution. Maybe there will be a second industrial revolution where the AIs design specialized tools so fast and powerful that they replace themselves in most tasks.

Let me provide a new argument for why we won't die in this particular way.

Long before the kid asks the AI to build a black hole, national laboratories would have done the same with far more resources. Why? Because an artificial black hole is the best possible energy source. It allows direct conversion of matter into energy -- just stream atoms into the black hole and collect the Hawking radiation.

Also, I believe the energy density you would need would require focused gamma rays and very large amounts of energy. AI might be smart, but it can't break the laws of physics. It would be easier to work with visible light, but that cannot focus as tightly and would thus need orders of magnitude more energy.

Generalizing, the problem is whether an aligned AI might perform a task that inadvertently causes damage. In the thought experiment, the AI was just trying to help a kid with a science experiment. However, I think that an aligned AI that is smart enough to solve the black hole problem would also be smart enough to see the consequences. Then, if the AI is truly aligned, it would not comply with the request.