How many legs do ten elephants have, if two of them are legless?

Re:How many legs do ten elephants have, if two of them are l... by insanitybit on Monday December 25, 2023 @05:16PM (#38766701)

No, it's actually executing that Python code. This is what allows an LLM (or an 'llm based system', I guess) to do something like "reverse <some uuid that has never been observed before>" - it can't just memorize the output and map it to the input because the output has literally never been observed. Instead, if it knows the algorithm for reversing a string, it can just use that and offload the execution to Python.

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by runarberg on Monday December 25, 2023 @06:12PM (#38767231)

That is even less impressive. I was thinking—like normal linear models—it would be capable of interpolation.

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by throwup238 on Monday December 25, 2023 @06:21PM (#38767302)

It is. It doesn’t even need an existing language. You can define your own psuedo language in the prompt and have ChatGPT “execute” it (works best with 4 nonturbo).

You can even combine your pseudo language with natural language. See the OP’s custom GPT and the comments here: https://news.ycombinator.com/item?id=38594521

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by insanitybit on Monday December 25, 2023 @07:50PM (#38767882)

That looks totally different. In the case of the Python code it is literally executing it by calling out to CPython.

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by no_identd on Monday December 25, 2023 @10:52PM (#38768772)

…got a source for that claim?

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by throwup238 on Monday December 25, 2023 @11:03PM (#38768816)

https://help.openai.com/en/articles/8302319-why-can-t-i-see-...

https://help.openai.com/en/articles/8437071-advanced-data-an...

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by labaron on Tuesday December 26, 2023 @07:53AM (#38771351)

I checked those links and didn’t see it mentioned that python code is actually executed. Could you quote the relevant part?

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by insanitybit on Tuesday December 26, 2023 @12:28PM (#38773771)

https://openai.com/blog/chatgpt-plugins#code-interpreter

>We provide our models with a working Python interpreter in a sandboxed, firewalled execution environment, along with some ephemeral disk space. Code run by our interpreter plugin is evaluated in a persistent session that is alive for the duration of a chat conversation (with an upper-bound timeout) and subsequent calls can build on top of each other. We support uploading files to the current conversation workspace and downloading the results of your work.

It really feels like I'm just googling for you, you had the feature name.

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by insanitybit on Monday December 25, 2023 @07:43PM (#38767836)

Why is it less impressive?

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by runarberg on Monday December 25, 2023 @08:08PM (#38767970)

I would say creating a model which is able to interpolate from training data in a way which produces an accurate output of a new input is a little impressive (if only as a neat party trick), however anybody can run a python interpreter on a server somewhere.

I’m sure there are use cases for this. But in the end it is only a simple feature added onto a—sometimes—marginally related service.

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by insanitybit on Monday December 25, 2023 @08:16PM (#38768029)

Hm, I don't think of it that way I guess. What the LLM is doing is generalizing a problem based on previous problems it has seen and then offloading the execution of that problem to a machine with some defined, specific semantics.

This is a lot more than a party trick. The model is able to describe the program it wants to execute and now it can accurately execute that - that it 'offloads' the work to a specialized program seems fine to me.

It's way more than a simple feature, this is enabling it to overcome one of the biggest limitations and criticisms of LLMs - it can answer questions it has never seen before.

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by nomel on Monday December 25, 2023 @05:24PM (#38766769)

There a code interpreter built in: https://www.pluralsight.com/resources/blog/data/chatgpt-code...

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by amelius on Monday December 25, 2023 @05:46PM (#38767009)

So it only works if the code is 100% correct. Which according to my experience doesn't happen often.

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by insanitybit on Monday December 25, 2023 @07:43PM (#38767838)

Most of the time it's just doing really simple stuff like this.

[ Parent | on HN ]

Re:How many legs do ten elephants have, if two of them are l... by nomel on Monday December 25, 2023 @06:32PM (#38767364)

Sure, don’t expect perfection, but it still works great.

[ Parent | on HN ]