That engraved post:

https://www.engraved.blog/building-a-virtual-machine-inside/

Showed that your initial assumption about the implausibility of the concept was completely and utterly incorrect.

You never explicitly mention how incorrect you were, but in the next paragraphs you proceed to sort of branch off into a different area.

The fascinating part about that engraved blog post wasn't the prime number code golf thing. The fascinating part was how chatGPT proceeded to emulate the entire internet, the proceeded to emulate the openAI website, then proceeded to emulate the chatGPT API on the openAI website, then in the most epic conclusion ever the author had the emulated chatGPT AGAIN recursively emulate a bash terminal.

Literally the article was a roller coaster ride to the end: A grand demonstration of a very trivial form of recursive self awareness in an AI from perspective of emulation.

You were not only epically wrong. But your subsequent paragraphs completely ignored the main point and decided to focus on some trivial prime number issue. Then afterwards you decided to branch off into some comment about how LLMs won't replace your coding job any time soon.

I think your post is interesting. Because it's highlighting the human psychology of self denial and bias. The OP article isn't even ABOUT AI replacing us, the linked blog post isn't even ABOUT the tiny prime number issue.

Instead your post both exists and is voted up because, humans, when faced with the impending possibility of a machine that can take over a skill many of us SW engineers have spent years honing into an art we can be proud of.... we have to deny that possibility because it's too frightening to face head on.

I can tell you this. I do not know future iterations of LLMs can take over our jobs. But to discount the possibility of it happening when the evidence is sitting in front of your face... is wildly irrational. We don't know if it will happen but given what's been going on in the past few months it is certainly a possibility that it CAN happen. The improvement in hallucinations from chatGPT to GPT4 occurred in mere months. What happens in a year? In two years?