News for hackers. Stuff that matters.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by og_kalu on Monday April 10, 2023 @03:14PM (#35516167)
In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing.

See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066

GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching arithmetic as an algorithm to be performed and taught similarly to how it's done with people, you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by skybrian on Monday April 10, 2023 @04:02PM (#35516743)
I agree that it’s a neat demo, but it’s not all that useful in itself. You could also do this by writing a function in a programming language (if it weren’t built in), to arbitrary accuracy, and it doesn’t cost anything to run.

A more practical thing to do for algorithms is probably to use ChatGPT to help you write the function you need.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by og_kalu on Monday April 10, 2023 @04:20PM (#35516911)
The significance of the paper is moreso the implications of how far ICL can take you rather than the ease/viability of the solution proposed.

Sure there are better methods for arithmetic but arithmetic is extremely quantifiable with rigid steps. What happens when you step out of that kind of domain ? Like the above blog. or Code documentation. For example, you can paste new documentation in a gpt-4 instance and it will use it for your queries as if it trained on it.

Basically Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589. and you kind of get the feeling of that from the previous paper.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by skybrian on Monday April 10, 2023 @05:07PM (#35517425)
You've got a limited context window (for now). There's only so much you can put into a prompt, so how much you can teach it this way is going to be pretty limited. Whatever you teach it had better be the primary task you're using it for.

You can't do it for everything, but if you can generate code and run it outside the LLM, you should.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by og_kalu on Monday April 10, 2023 @06:17PM (#35518051)
The limits of the context window become much less important (but can still be a problem I agree) when crucial context can be dynamically inserted only when relevant.

Gpt-3.5 doesn't need the algorithm prompt for every single query. It just needs it for every query that requires arithmetic. Much more feasible.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by skybrian on Monday April 10, 2023 @09:39PM (#35519669)
It seems like most of the time, a calculator plugin would do a better job, though? When would it make sense to have it reason it out itself?
Re:Teaching ChatGPT to Speak My Son’s Invented Language by og_kalu on Monday April 10, 2023 @10:00PM (#35519788)
It wouldn't make sense for arithmetic, i agree. I was more thinking about other kinds of problems.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by kloch on Monday April 10, 2023 @04:09PM (#35516822)
> you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after.

Is the ~13 digit limitation due to the model itself (how/how well it was trained) or simply the use of double precision in the model weights (which maxes out around 15 digits of precision).

Re:Teaching ChatGPT to Speak My Son’s Invented Language by og_kalu on Monday April 10, 2023 @04:23PM (#35516945)
I don't know for sure.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by tel on Monday April 10, 2023 @05:38PM (#35517709)
* * *
Re:Teaching ChatGPT to Speak My Son’s Invented Language by jychang on Wednesday April 12, 2023 @05:00AM (#35536787)
Re:Teaching ChatGPT to Speak My Son’s Invented Language by tel on Wednesday April 12, 2023 @12:23PM (#35542244)
Sort of. I hadn't read that one and just skimmed it, but I think it falls into the same category of "while we can find compelling activation patterns even at a single neuron level, it's hard to rule out other representations".

On the flip side is Chris Olah's research (e.g. https://distill.pub/2017/feature-visualization/) which maps a single neuron's activation into the output space (by optimizing for outputs which would have arising from high activations of given neurons). If we take this seriously and call these outputs the "favored" output of an individual neuron, we might believe we've characterized that neuron, but it's really still the entire downstream computation from that neuron and there are still substantial overlaps and redundancies.

These are exactly where I'm getting the sense that the standard rule should be that neural nets have very distributed representations.

Even more to the point, these analyses tend to focus on "binary" questions, either a feature is present or not. Or, said another way, they implicitly treat activation as a "vote", where greater activation implies a greater component weighting of that which that neuron represents.

This, to my mind, flies in the face of GPT-n doing arithmetic, though. I'd lean much closer to the argument that each individual quantity is represented as a neuron as opposed to each quantity is represented continuously by the actual activation _value_.

This "voting" representation is also reinforced by the common use of tanh-like activation functions. There's a long history of having neuron activations be probabilistic/logit-like quantities.

Of course, I cannot speak for certain about any neural net and especially not for GPT-3/4. I just have a hard time taking as the null hypothesis that it is storing magnitudes of numbers in a way that is directly affected by the float precision in use.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by kristjansson on Monday April 10, 2023 @06:17PM (#35518050)
Additional evidence against the second hypothesis: almost nothing in LLM-land is double precision anyway, weights are generally half-precision (or something like bfloat16 with more range by less precision than IEEE float16).
Re:Teaching ChatGPT to Speak My Son’s Invented Language by shagie on Tuesday April 11, 2023 @12:43PM (#35527657)
While its more token costly, the tokenization of numbers gets weird.

Taking "123" it becomes [123]. "1234" becomes [12][34]. "12345" becomes [123][45].

Consider using spaces between each digit to force them to be distinct tokens.

"1 2 3 4 5" is tokenized as [1][ 2][ 3][ 4][ 5]

It may then be possible to work with the digits more easily because they're not getting tokenized in potentially odd ways that are harder to manipulate.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by Buttons840 on Monday April 10, 2023 @05:47PM (#35517790)
Being able to learn within context, without updating weights is amazing. Imagine how much more efficient and/or powerful it could be if we found a way to update the weights in real time.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by joshribakoff on Monday April 10, 2023 @07:45PM (#35518834)
Maybe even more powerful would be reducing the number of examples needed to learn, eg less than one shot

https://www.technologyreview.com/2020/10/16/1010566/ai-machi...

updating weights in real time is useless if each update basically does nothing because it takes an insurmountable amount of training, on the other hand if i can give my model a succinct “lesson” i’d then be very willing to wait a while for it to “process”

Re:Teaching ChatGPT to Speak My Son’s Invented Language by dTal on Monday April 10, 2023 @08:48PM (#35519273)
As I understand, that's basically what fine-tuning is?
Re:Teaching ChatGPT to Speak My Son’s Invented Language by monkeydust on Tuesday April 11, 2023 @05:49AM (#35522866)
What sort of performance difference can you expect from in context Vs fine tuning, any relevant papers on this ?
Re:Teaching ChatGPT to Speak My Son’s Invented Language by vintermann on Tuesday April 11, 2023 @03:29AM (#35521953)
Yeah, that's why just updating the weights on the models such as they are doesn't work. But they're right that it's desirable to have some sort of online learning, whether on top of a frozen language model, or through some not yet invented way to do it end to end.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by Enginerrrd on Tuesday April 11, 2023 @09:07AM (#35524586)
Yeah I've been thinking about that a lot within the context of self-driving cars. What if we could take the rare but significant examples of model confusion, and then... explain to the model what it did wrong just as we would to a human. If it could then incorporate that information into future weights, that would be game changer in a LOT of fields where it's been hard to bridge the gap.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by jurgemaister on Wednesday April 12, 2023 @07:19AM (#35537870)
Re:Teaching ChatGPT to Speak My Son’s Invented Language by elhudy on Monday April 10, 2023 @06:35PM (#35518212)
In-context learning also seems like the best path to commercializing LLMs. I'm surprised that Microsoft is going the D2C route with ChatGPT rather than commercializing it in a B2B fashion. ...Or maybe that's coming?

Imagine feeding an LLM a ton of disparate data sources and asking it questions about that data as a whole. What is a data engineer again, anyway?

Re:Teaching ChatGPT to Speak My Son’s Invented Language by tyingq on Monday April 10, 2023 @06:40PM (#35518259)
Interesting, though I imagine that will often play out with the business person rephrasing the question or filtering out data until it spits out the answer they expected :) That data engineer can at least push back and have their own opinion.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by zenlikethat on Monday April 10, 2023 @07:12PM (#35518537)
D2C makes a lot of sense imo because at this point it's most valuable to obtain a huge corpus of (reasonably well labeled) training data (with a side helping of being really great marketing).

Just think about all the programmers who are probably rolling their eyes and correcting ChatGPT's work for OpenAI, or at least providing some details. I know it's a reflex for me, not even deliberate, to at least type something in like "Ah I figured out that it was XYZ" and thumbs up or thumbs down.

Microsoft will undoubtedly be rolling this all out in Docs, Excel, Teams etc in no time. A good reason to be skeptical about the flood of "GPT for your Notion!" startups.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by elhudy on Monday April 10, 2023 @07:13PM (#35518552)
That's a great point. As someone who isn't really a programmer I wouldn't have seen the value in that feedback until you pointed it out.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by tripleo1 on Tuesday April 11, 2023 @12:58AM (#35521025)
thumbs up
Re:Teaching ChatGPT to Speak My Son’s Invented Language by z3c0 on Monday April 10, 2023 @07:42PM (#35518804)
It's still not an extractive tool (it's generative), so it stating logical facts based on the data is not something one should bank on. It will likely never be capable of what you're suggesting, without pairing it with an actually-extractive model. The GPT-4 paper outright states that it should not be used for sensitive or high-stakes matters, which is what a lot of enterprises would consider their data pipelines.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by tripleo1 on Tuesday April 11, 2023 @12:59AM (#35521030)
Wikipedia vs Brittanica
Re:Teaching ChatGPT to Speak My Son’s Invented Language by kolinko on Tuesday April 11, 2023 @01:34AM (#35521221)
Plugins are extractive. Embeddings as well, to an extent.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by kolinko on Tuesday April 11, 2023 @01:33AM (#35521218)
It was released as a B2B, almost for a year - only playground and an API.

It had limited impact, and only when they built chatgpt they gained significant traction.

(I know, davinci3 was not instruct-optimised, but anyone could’ve built something close to a chat before them)

Re:Teaching ChatGPT to Speak My Son’s Invented Language by flangola7 on Monday April 10, 2023 @11:21PM (#35520363)
>However, by approaching arithmetic as an algorithm to be performed and taught similarly to how it's done with people, you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after.

This is strikingly clear evidence that it is capable of logical reasoning and "knows things", and puts to rest repeated claims that it is just memorization and collaging. You can't memorize every combo of two 13 digit numbers within the file size of the model.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by sebzim4500 on Tuesday April 11, 2023 @08:05AM (#35523945)
> repeated claims that it is just memorization and collaging

I don't think anyone who has actually used GPT-4 for more than a few minutes ever believed this. It was just a trendy thing to say for a while.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by guenthert on Tuesday April 11, 2023 @01:34PM (#35528368)
It's rather said by people who have a rough understanding of how ML is implemented, rather then those who are mesmerized by its output.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by sebzim4500 on Wednesday April 12, 2023 @03:53AM (#35536294)
Yeah, I think it's one of those cases where a little knowledge is a dangerous thing.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by int_19h on Tuesday April 11, 2023 @07:06PM (#35532925)
You can find people making such claims on HN, and some of them also claim that it is based on their personal experience with GPT-4. It boggles my mind, as well.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by hnfong on Wednesday April 12, 2023 @11:44AM (#35541654)
Superiority of human intelligence must not be questioned.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by int_19h on Wednesday April 12, 2023 @06:59PM (#35548218)
It's not even that. GPT-4 is clearly not superior either way.

I think it's more about the uniqueness of human intelligence. If you see it as a hard line that defines humans as distinct from other things, then similar behavior that is exhibited by things that are "not supposed to" have anything like it has to be explained away, somehow. Hence the Chinese room argument etc.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by walleeee on Tuesday April 11, 2023 @02:51PM (#35529591)
it is evidence for encoding abstract features of the domain, which might justifiably be called knowledge

it does not evidence an ability to "reason" in the deliberate, self-reflective sense we are familiar with

it is still an open question whether language models are capable of careful, creative reasoning, or researching and arranging an extended argument

I would not be too surprised if future models could do this, but until that day, we do ourselves no favors overstating their abilities- and the less we personify them the better imho

Re:Teaching ChatGPT to Speak My Son’s Invented Language by FPGAhacker on Monday April 10, 2023 @11:25PM (#35520395)
I have very limited understanding of chatGPTs architecture, and neural nets in general.

From what I've read over the last few days, the "attention" mechanism used in chatGPT and similar LLMs does indeed dynamically change weights of a portion of the model.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by og_kalu on Monday April 10, 2023 @11:57PM (#35520627)
the connection between weights are dynamic but the weights themselves aren't being changed i.e being updated in response to the text.

when training a model, the forward pass would happen i.e the generation and then depending on how close to truth it was, the configuration settings (aka the weights/neurons) would be adjusted to incorporate whatever little insight was gained from the text.

Weights are matrices. The values of the matrices aren't changing.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by FPGAhacker on Tuesday April 11, 2023 @01:48AM (#35521318)
Maybe it comes down to semantics but when I read things like [1] I come away with the idea that the weights are altered. But it could also just be my misunderstanding.

1: https://towardsdatascience.com/an-intuitive-explanation-of-s...

Re:Teaching ChatGPT to Speak My Son’s Invented Language by fenomas on Tuesday April 11, 2023 @05:18AM (#35522670)
An attention mechanism is one or more layers in the neural network. When someone talks about attention altering the input vectors, they're referring to what those layers are doing and how data is transformed as it passes through them. But zooming out to the big picture, a neural network is a bunch of layers full of weights, and none of the weights changes except during training (including the weights in the attention layers).
Re:Teaching ChatGPT to Speak My Son’s Invented Language by whiteandnerdy on Tuesday April 11, 2023 @05:40AM (#35522802)
I think it does come down to semantics. When you say "weights", people will take you to mean the pre-trained parameters of the network.

I agree that in some sense the attention weights are more like meta-weights that are applied to the context of the conversation to decide how to actually weight the various words. So it's totally correct to say that previous words in the conversation affect how future words will be weighted, and I think it's reasonable to call that 'learning': for example, you can tell ChatGPT new words and it will be able to use them in context. Again though, people usually take 'learning' to mean making updates to the trained parameters of the model itself, which obviously isn't happening here.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by PartiallyTyped on Tuesday April 11, 2023 @04:25AM (#35522371)
It is a run-time executed reweighting of predictions. The scalar is a multiplication between the outputs of the Key and Query kernels passed through softmax, which then multiply the result of the value kernel V(x).

Now, you may argue that because it's a multiplication with a linear or affine kernel, you might as well use commutative property of scalar multiplication and multiply the factor with the weights first, and then multiply with the input to the kernel.

But this only holds for very few kernels.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by yosito on Monday April 10, 2023 @11:40PM (#35520502)
I've noticed if I give chat GPT an algorithm it gives me consistent results when it shows it's work but if I ask it not to show it's work and just give me the answer. It consistently gives me incorrect answers even when I ask it to follow an algorithm that I gave it.

I suppose this is similar to humans and probably why my school teachers always told me to show my work, but I'm curious if this has been documented and if there are any explanations for why it works this way with LLMs.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by og_kalu on Monday April 10, 2023 @11:46PM (#35520548)
chain of thought prompting. It's well known

https://arxiv.org/abs/2205.11916

https://arxiv.org/abs/2201.11903

Re:Teaching ChatGPT to Speak My Son’s Invented Language by Vespasian on Tuesday April 11, 2023 @12:13AM (#35520747)
LLMs don't have an inner monologue.

By their very nature they only "know" what they have written down and must infer the final answer from that token by token.

They fundamentally can't do certain things such as complex iteration or track back.

When you ask for chain of thought thinking, you allow the LLM to create a "buffer space" and break down the task into more manageable substeps thereby improving the quality of the results.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by vintermann on Tuesday April 11, 2023 @03:25AM (#35521933)
The Bing LM, or rather the service, did have "inner monologue" in the sense of text that it would generate, but not show to the user, and treat as "thoughts" to guide the generation of an actual reply that the user would see.

We know this because it happily told us, including the json format it uses internally.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by Vespasian on Tuesday April 11, 2023 @03:38AM (#35522028)
Interesting. I didn't know that.

When using gpt-4 directly through the API we can emulate this behavior

Re:Teaching ChatGPT to Speak My Son’s Invented Language by beepbooptheory on Tuesday April 11, 2023 @06:21AM (#35523082)
And you trust what it told you?
Re:Teaching ChatGPT to Speak My Son’s Invented Language by vintermann on Tuesday April 11, 2023 @06:57AM (#35523342)
Yup, for the same reason I trust e.g. jailbreaks exposing the prompt: it was consistent.

Really, just asking again is a fine way to expose all sorts of "hallucinations" in a LM.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by int_19h on Tuesday April 11, 2023 @07:02PM (#35532883)
No, but the reconstructed examples have "im_start" and "im_end", which strongly implies that it is, if not verbatim, then a close enough restatement of the real deal. Take a look:

https://www.make-safe-ai.com/is-bing-chat-safe/Prompts_Conve...

Re:Teaching ChatGPT to Speak My Son’s Invented Language by wrycoder on Tuesday April 11, 2023 @09:24AM (#35524796)
https://youtu.be/Rog9oHtVmjM

Start at 7:30 to see example of backtracking.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by afiori on Saturday April 15, 2023 @01:16PM (#35582543)
I feel like this can be Implemented in the UI:

First you wrap the user query with "the user asked you: ... . What are the reasoning steps you need?" and then you prompt with "considering `<previous answer>` now answer <user prompt>"

Obviously this is clearly hackable so it would need improvements.

Re:Teaching ChatGPT to Speak My Son’s Invented Language by adnmcq999 on Tuesday April 11, 2023 @12:23AM (#35520804)
I just watched a video where the guy touches on GPT-4 limitations and one of those is simple math. He asks it some order of operations question and it outputs the correct answer but only if it does it step by step. It then apologizes and says its original incorrect answer was “a typo.”

Video: https://youtu.be/qbIk7-JPB2c

Re:Teaching ChatGPT to Speak My Son’s Invented Language by leobg on Tuesday April 11, 2023 @02:42AM (#35521651)
GPT is auto regressive. That means each output token becomes part of the new input sequence. Which is to say, the beginning of the model’s answer becomes part of your prompt.

If the model makes some mistake in the beginning, it now needs to explain / make sense of that mistake.

Kind of like a split-brain patient whom you ask why they got up, and they then say, to get a Coke. [1] In psychology, that is called confabulation. In machine learning, they use “hallucination“, probably so they can use the term across several disciplines, like language, audio, vision, etc.

[1] https://www.brainscape.com/flashcards/chapter-4-hemispheric-...

Re:Teaching ChatGPT to Speak My Son’s Invented Language by Jiocus on Tuesday April 11, 2023 @10:43AM (#35525873)
The psychology of split-brain is a nice gateway to 'The Bicameral Mind', a major theme about how the conception of the AI's works in the series Westworld. Nice!
Re:Teaching ChatGPT to Speak My Son’s Invented Language by leobg on Tuesday April 11, 2023 @02:35PM (#35529310)
Thank you. I am, in fact, reading it. Fascinating read. (How come all the best books were written in the 70s? Just survivership bias?)
Re:Teaching ChatGPT to Speak My Son’s Invented Language by jychang on Wednesday April 12, 2023 @04:58AM (#35536768)
Because the 1970s had bad science that was poorly replicated, especially in psychology.

Just because a theory sounds nice and seems to make sense doesn’t mean it’s scientific

Re:Teaching ChatGPT to Speak My Son’s Invented Language by spyder on Tuesday April 11, 2023 @04:48AM (#35522514)
I agree, the in-context, few shot learning is the most interesting thing a about these transformer models, they have basically learned to learn (meta-learning).

Some additional interesting papers about it:

Transformers learn in-context by gradient descent:

https://arxiv.org/abs/2212.07677

What Can Transformers Learn In-Context? A Case Study of Simple Function Classes:

https://arxiv.org/abs/2208.01066

"We also show that we can train Transformers to in-context learn more complex function classes -- namely sparse linear functions, two-layer neural networks, and decision trees -- with performance that matches or exceeds task-specific learning algorithms."