So on the one hand, these newly publicized models can render convincing representations of realities we used to get from deterministic processes. On the other hand, it's probabilistic and fails to conform to logic quite often, and in a confident way.

We're building systems capable of programing computers non-deterministically. I think this is huge. But not because ChatGPT23 will be a CEO or a politician. But because this is a paradigm shift in compute similar to moving from integrator machines to general computers. I don't think LLMs will make programmers obsolete. I think large enough models will make programming something completely different from what it is now.

The days of sequencing tokens for compilers/interpreters seem to be drawing to an end as the dominant way of specifying software products.

I'm sort of glad to be retiring soon. I have a feeling everything I enjoy about programming is going to be going away.

I get the feeling. But I've always enjoyed the abstract puzzles more then anything. Computers attracted me as a form of very complex abstract puzzles.

But when it comes down to it everything in life is just nth dimensional tensor puzzles. What I really cherish computers for giving me is fast and clear feedback.

I am 10-15 years away from retirement and have been giving a lot of thought to how to navigate the remainder of my career in light of recent AI developments lately.

This disruption does appear to be different than others prior. As it is not a narrow disruption with limited boundaries for which we can plan and organize our lives for some stabilization period to follow.

Instead, it is constant and accelerating destabilization. Hundreds of AI projects attempting to ride this accelerating wave were essentially just made obsolete yesterday - https://www.youtube.com/watch?v=DH-2BHDYNfk

I feel the excitement is going to very soon turn into frustration of attempting to remain relevant ahead of the accelerating technological curve. Humans need periods of stabilization to plan and reason about their lives.

The LLM can act as a global cache for common solutions to common problems, with the ability to perform the integration work necessary to apply them.

That prime number example is a little bit like when you put a functools.lru_cache decorator on a function in Python. It's faster than computing the function call because it's able to recall the return value for the parameters from the cache "memory".

Of course, many skilled programmers are also mainly used a cache for common solutions to common problems organizations have in the programming domain. As humans we can derive satisfaction from being able to tackle the same task others can, as a confirmation of our own progress. We like "doing it ourselves". But globally that's not super interesting if lots of people are constantly re-developing the same solutions to the same problems for their own benefit. I guess that's the push-and-pull (and the anxiety generator).

>The LLM can act as a global cache for common solutions to common problems, with the ability to perform the integration work necessary to apply them.

In my opinion Stack Overflow does a fine job at that. And it's transparent in terms of solutions proposed being voted on and discussed. Turning that into sausage advice is a downgrade.

>But globally that's not super interesting if lots of people are constantly re-developing the same solutions to the same problems for their own benefit.

I'd argue this is how we train individuals thus globally quite relevant.

So it sounds like the mechanism is something like stochastic memoisation?

I don't think "memoisation" is an accurate word for this; it implies doing the computation once, and storing the result in cache to return later. It's more like replacing your Python code with a SQL query of the LLM's understanding of what it's supposed to do, which may or may not be correct, and executing it on its "database" of knowledge, which may or may not have the correct data.

this is new to me but.. a quick read of Wikipedia [1] later, it appears that this decades-old method takes a goal, and then uses branching and recording to find paths that differ the least from the goal. The article mentions that in practice, the curse of dimensionality is so bad here that approximations are often used in practice. Does that capture it?

[1] https://en.wikipedia.org/wiki/Stochastic_dynamic_programming

> This caught my attention as I found it implausible

That engraved post:

https://www.engraved.blog/building-a-virtual-machine-inside/

Showed that your initial assumption about the implausibility of the concept was completely and utterly incorrect.

You never explicitly mention how incorrect you were, but in the next paragraphs you proceed to sort of branch off into a different area.

The fascinating part about that engraved blog post wasn't the prime number code golf thing. The fascinating part was how chatGPT proceeded to emulate the entire internet, the proceeded to emulate the openAI website, then proceeded to emulate the chatGPT API on the openAI website, then in the most epic conclusion ever the author had the emulated chatGPT AGAIN recursively emulate a bash terminal.

Literally the article was a roller coaster ride to the end: A grand demonstration of a very trivial form of recursive self awareness in an AI from perspective of emulation.

You were not only epically wrong. But your subsequent paragraphs completely ignored the main point and decided to focus on some trivial prime number issue. Then afterwards you decided to branch off into some comment about how LLMs won't replace your coding job any time soon.

I think your post is interesting. Because it's highlighting the human psychology of self denial and bias. The OP article isn't even ABOUT AI replacing us, the linked blog post isn't even ABOUT the tiny prime number issue.

Instead your post both exists and is voted up because, humans, when faced with the impending possibility of a machine that can take over a skill many of us SW engineers have spent years honing into an art we can be proud of.... we have to deny that possibility because it's too frightening to face head on.

I can tell you this. I do not know future iterations of LLMs can take over our jobs. But to discount the possibility of it happening when the evidence is sitting in front of your face... is wildly irrational. We don't know if it will happen but given what's been going on in the past few months it is certainly a possibility that it CAN happen. The improvement in hallucinations from chatGPT to GPT4 occurred in mere months. What happens in a year? In two years?

Equally interesting is the psychology of people who take the time to write long posts borderline gloating about tools making software engineers obsolete.

Except I didn't write that and I will quote myself:

>"I can tell you this. I do not know future iterations of LLMs can take over our jobs"

I wrote that I don't know which is the best answer we all have at this point.

Given the evidence, completely denying it, as many have is simply not realistic.

Okay, but you have another post in this thread with:

> When these LLMs get normalized probably 5 years from now I'm going go back to these old threads and contact these people who are in self denial and throw it in their face. I'll just link this comment and be like I TOLD YOU, I TOLD YOU, YOU WERE WRONG.

Buddy, didn't need to stalk my posts and flag. The offense is a matter of opinion and I do respond with appropriate changes so your feelings aren't hurt.

I didn't flag your posts, friend. Even if I dislike the tone I get the impression you're being sincere, and I think they add to the conversation.

My bad. It's the caps. I'll remove the caps. Just picture me saying it a nicer way.

Gloating over people being wrong is rarely nice

Not if it's said in a nice way. The commenter called LLMs a freaking auto complete. That's not nice either, but nice enough for HN. So I figured a little subtle gloating is deserved and well within the bounds of etiquette here on HN.

Well insulting the capacity of an LLM is much different than gloating over someone being wrong about their livelihood.

Rude.

Rude but real. The comment is off topic and part of a common attitude of always attempting to diminish the capabilities of LLMs to what end?

What's most interesting to me, is that this is how I would expect a human to approach the problem if presented with the code and asked for the output.

The LLM didn't run the code, it tried to predict the output based on its knowledge of python and primes.

When I was a bored and under-challenged student in the early days of university, one of my tests during a programming exam was to write a program that spit out the factorial of an input integer.

For shits and giggles, I just just wrote a series of `if...then` statements for 0 to 5, and only actually calculated the factorial for inputs >=6. I passed after the examiner just input 3 and 5 and was satisfied that the output was correct. Oops.

Doing a table lookup for common cases and computing for the less common ones is perfectly valid!

And for some problems it's even efficient. Not factorial, though.

Oh you're right, hive mind, it's efficient. Sorry.

Not 100% sure but I believe this is how we landed the lunar module on the moon the first time...tan/arctan/both were too hard to compute on the processors of those days so they discretized into half angles & stored the tangents in a lookup table.

That's how early calculators worked, and some fast math libraries today too.

The Pentium FDIV bug was due to a bad value in a lookup table.

This is the way. Next, you could cache those values for >=6 when you compute them and also use the previously cached values for sub-problem solutions.

Sorry, did it try to predict the output, or did it only predict the output based on what it has seen people say about similar code online?

Is your objection to the use of the word "try"?

The difference is "did it study the code and inputs/outputs" vs "it studied comments about the code"

What would it do if the training data was only code? Only code+inputs+outputs? Only function signature+inputs/outputs? Only signature+comments?

Etc.

The second one

I think its knowledge of python can only come from comments about code, since it can't execute the python or learn from trying code like a human would.

That doesn't change the fact that its predicting the output based on its knowledge of python though.

So, yes you're right, but at the same time it doesn't change anything IMO.

So here are a few screenshots that I personally took after telling it that it was a linux terminal:

Correctly produces a convincing output for having decoded the base64-encoding of "ping google.com" and then piping it to bash: https://media.infosec.exchange/infosecmedia/media_attachment...

Similar command, but with a garbage domain it hasn't seen before, and a less well-known domain. It produced convincing output in both cases: https://media.infosec.exchange/infosecmedia/media_attachment...

Having it just output a base64 decoding of an obviously unique string. Fascinatingly, it tried to correct typos that I intentionally included: https://media.infosec.exchange/infosecmedia/media_attachment...

This was also pretty cool -- ask it to start a webserver then curl it: https://media.infosec.exchange/infosecmedia/media_attachment...

Telling it that it is a python interpreter and calling self-defined functions: https://media.infosec.exchange/infosecmedia/media_attachment...

A slightly more complicated function: https://media.infosec.exchange/infosecmedia/media_attachment...

I did a few more experiments including generating large factorial numbers that took a long time on my laptop but it responded accurately to a much larger length than my laptop could do (though these were only accurate to the first few hundred digits)

Lol why does ChatGPT hit ^C? Do you think it’s getting bored of waiting to respond to you and decides that’s enough time spent on an answer?

Edit: the base64 decoding examples are terrifying. I have no idea how that works.

it's probably hitting 'caret' 'C' except when it's using Emacs when it types C-C

yep I tried asking chatgpt to optimize some SQL queries with a heavy amount of full outer joins. The optimization I was trying to push it to was specifically adding an index on a column or filtering the first table that we join on but it kept creating SQL subqueries and switching the full joins to left joins no matter what I told it and the sql errors I sent it

Large language models are storytellers. To write a story about someone using a computer terminal, there are things it’s helpful to know, but there are easier ways than simulating a computer.

Since we don’t know how it works, we should be open to the possibility that it’s using all sorts of storytelling techniques.

Gpt is never calculating anything!!

Its speed as compared to the laptop - I assume it was running online, on OpenAI’s hardware. That hardware is quite extensive, I thought. I believe I’ve seen the cost of one (not ‘search’, what’s the word here, ‘output’?) is something like 1-10 cents.

I was personally quite appalled that so many people believed this was what it was doing, but I suppose since the source was a DeepMind researcher, it lends a lot of credibility.

By the same reasoning, I hope people do realise that there are computations that are just impossible given the finite depth in a Transformer. The only possible way to overcome this in the current paradigm is the Chain-of-Thought related methods.

Yeah, I tried asking GPT-4 to write a simple Python program to find the millionth prime. It wrote the naive approach where you loop over numbers and for each number, check all of the possible divisors up to its square root, and then keep track of how many primes you've found. Obviously, this takes quite a while to execute for N = a million. And yet, if you ask GPT-4 what the output of the code will be, as if my magic (memorization) it immediately spits out 15485863. It makes it a lot more clear that it's not actually simulating the code.