It was no problem finding an unprotected home directory with a solution in it, but we found the program didn't work. In the end we had to not only modify the program enough to not get caught, but we had fix the bugs in it.

Little did I know what good preparation this would be for my career in software development!

Devastated by the impact of a $100 million project failure I took an underpaid job for a small but proud web development shop based at a Superfund site were I completed roughly 20 projects that other programmers had started in about 9 months. It was the most acute example of something I'd experienced a lot in my career, both before and after, where somebody, anybody from a complete fresher to a certified genius to somebody getting a masters in A.I. because they really needed the intelligence, built something they couldn't finish and left behind a product that looked promising but needed serious rework to get it in front of customers. (... Then we ran out of projects, I cracked, and two days later got a job at the other web development shop that was landing all the new contracts that we were failing to get.)

I see GPT-3 as that fresher programmer who can make things that look promising to management but in the end turn out to need a huge amount of rework to put in front of customers. For a time I was greatly resentful that somebody would seem to do the "20% of the work that gets 80%" of the results and it seemed I'd do the "80% of the work that gets 20% of the results" and have people complain I took to long to do things, even during my annus mirabalus at Spider Graphics or many other times I'd saved a project that had been circling the drain for years.

GPT-3 has a hypnotic ability to get away with making mistakes which I think is a product of it being trained to produce the token with the highest probability. Like Andy Warhol, it is actively anti-creative.

Fixing the hard-to-find mistakes that it makes will be a maddening job and people will always be looking for ways to push the bubble out from under the rug and not realize the machine they are trying to build is impossible for fundamental logical reasons. I think of the dialogs of Achilles and the Tortoise from

https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach

where they are trying to build impossible machines and repeatedly failing because they have no idea that what they're trying to do is impossible. I've had people say GEB is a critique of the old symbolic AI but neural networks don't repeal the fundamental results of mathematical logic and computer science.

Sure, you can escape Gödel's theorem by building a system that doesn't get the right answer but then you have a system that doesn't get the right answer.