A more practical thing to do for algorithms is probably to use ChatGPT to help you write the function you need.
Sure there are better methods for arithmetic but arithmetic is extremely quantifiable with rigid steps. What happens when you step out of that kind of domain ? Like the above blog. or Code documentation. For example, you can paste new documentation in a gpt-4 instance and it will use it for your queries as if it trained on it.
Basically Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589. and you kind of get the feeling of that from the previous paper.
You can't do it for everything, but if you can generate code and run it outside the LLM, you should.
Gpt-3.5 doesn't need the algorithm prompt for every single query. It just needs it for every query that requires arithmetic. Much more feasible.
Is the ~13 digit limitation due to the model itself (how/how well it was trained) or simply the use of double precision in the model weights (which maxes out around 15 digits of precision).
https://www.lesswrong.com/posts/cgqh99SHsCv3jJYDS/we-found-a...
On the flip side is Chris Olah's research (e.g. https://distill.pub/2017/feature-visualization/) which maps a single neuron's activation into the output space (by optimizing for outputs which would have arising from high activations of given neurons). If we take this seriously and call these outputs the "favored" output of an individual neuron, we might believe we've characterized that neuron, but it's really still the entire downstream computation from that neuron and there are still substantial overlaps and redundancies.
These are exactly where I'm getting the sense that the standard rule should be that neural nets have very distributed representations.
Even more to the point, these analyses tend to focus on "binary" questions, either a feature is present or not. Or, said another way, they implicitly treat activation as a "vote", where greater activation implies a greater component weighting of that which that neuron represents.
This, to my mind, flies in the face of GPT-n doing arithmetic, though. I'd lean much closer to the argument that each individual quantity is represented as a neuron as opposed to each quantity is represented continuously by the actual activation _value_.
This "voting" representation is also reinforced by the common use of tanh-like activation functions. There's a long history of having neuron activations be probabilistic/logit-like quantities.
Of course, I cannot speak for certain about any neural net and especially not for GPT-3/4. I just have a hard time taking as the null hypothesis that it is storing magnitudes of numbers in a way that is directly affected by the float precision in use.
Taking "123" it becomes [123]. "1234" becomes [12][34]. "12345" becomes [123][45].
Consider using spaces between each digit to force them to be distinct tokens.
"1 2 3 4 5" is tokenized as [1][ 2][ 3][ 4][ 5]
It may then be possible to work with the digits more easily because they're not getting tokenized in potentially odd ways that are harder to manipulate.
https://www.technologyreview.com/2020/10/16/1010566/ai-machi...
updating weights in real time is useless if each update basically does nothing because it takes an insurmountable amount of training, on the other hand if i can give my model a succinct “lesson” i’d then be very willing to wait a while for it to “process”
[1]https://www.forbes.com/sites/bradtempleton/2019/04/29/teslas...
Imagine feeding an LLM a ton of disparate data sources and asking it questions about that data as a whole. What is a data engineer again, anyway?
Just think about all the programmers who are probably rolling their eyes and correcting ChatGPT's work for OpenAI, or at least providing some details. I know it's a reflex for me, not even deliberate, to at least type something in like "Ah I figured out that it was XYZ" and thumbs up or thumbs down.
Microsoft will undoubtedly be rolling this all out in Docs, Excel, Teams etc in no time. A good reason to be skeptical about the flood of "GPT for your Notion!" startups.
It had limited impact, and only when they built chatgpt they gained significant traction.
(I know, davinci3 was not instruct-optimised, but anyone could’ve built something close to a chat before them)
This is strikingly clear evidence that it is capable of logical reasoning and "knows things", and puts to rest repeated claims that it is just memorization and collaging. You can't memorize every combo of two 13 digit numbers within the file size of the model.
I don't think anyone who has actually used GPT-4 for more than a few minutes ever believed this. It was just a trendy thing to say for a while.
I think it's more about the uniqueness of human intelligence. If you see it as a hard line that defines humans as distinct from other things, then similar behavior that is exhibited by things that are "not supposed to" have anything like it has to be explained away, somehow. Hence the Chinese room argument etc.
it does not evidence an ability to "reason" in the deliberate, self-reflective sense we are familiar with
it is still an open question whether language models are capable of careful, creative reasoning, or researching and arranging an extended argument
I would not be too surprised if future models could do this, but until that day, we do ourselves no favors overstating their abilities- and the less we personify them the better imho
From what I've read over the last few days, the "attention" mechanism used in chatGPT and similar LLMs does indeed dynamically change weights of a portion of the model.
when training a model, the forward pass would happen i.e the generation and then depending on how close to truth it was, the configuration settings (aka the weights/neurons) would be adjusted to incorporate whatever little insight was gained from the text.
Weights are matrices. The values of the matrices aren't changing.
1: https://towardsdatascience.com/an-intuitive-explanation-of-s...
I agree that in some sense the attention weights are more like meta-weights that are applied to the context of the conversation to decide how to actually weight the various words. So it's totally correct to say that previous words in the conversation affect how future words will be weighted, and I think it's reasonable to call that 'learning': for example, you can tell ChatGPT new words and it will be able to use them in context. Again though, people usually take 'learning' to mean making updates to the trained parameters of the model itself, which obviously isn't happening here.
Now, you may argue that because it's a multiplication with a linear or affine kernel, you might as well use commutative property of scalar multiplication and multiply the factor with the weights first, and then multiply with the input to the kernel.
But this only holds for very few kernels.
I suppose this is similar to humans and probably why my school teachers always told me to show my work, but I'm curious if this has been documented and if there are any explanations for why it works this way with LLMs.
By their very nature they only "know" what they have written down and must infer the final answer from that token by token.
They fundamentally can't do certain things such as complex iteration or track back.
When you ask for chain of thought thinking, you allow the LLM to create a "buffer space" and break down the task into more manageable substeps thereby improving the quality of the results.
We know this because it happily told us, including the json format it uses internally.
When using gpt-4 directly through the API we can emulate this behavior
Really, just asking again is a fine way to expose all sorts of "hallucinations" in a LM.
https://www.make-safe-ai.com/is-bing-chat-safe/Prompts_Conve...
Start at 7:30 to see example of backtracking.
First you wrap the user query with "the user asked you: ... . What are the reasoning steps you need?" and then you prompt with "considering `<previous answer>` now answer <user prompt>"
Obviously this is clearly hackable so it would need improvements.
Video: https://youtu.be/qbIk7-JPB2c
If the model makes some mistake in the beginning, it now needs to explain / make sense of that mistake.
Kind of like a split-brain patient whom you ask why they got up, and they then say, to get a Coke. [1] In psychology, that is called confabulation. In machine learning, they use “hallucination“, probably so they can use the term across several disciplines, like language, audio, vision, etc.
[1] https://www.brainscape.com/flashcards/chapter-4-hemispheric-...
Just because a theory sounds nice and seems to make sense doesn’t mean it’s scientific
Some additional interesting papers about it:
Transformers learn in-context by gradient descent:
https://arxiv.org/abs/2212.07677
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes:
https://arxiv.org/abs/2208.01066
"We also show that we can train Transformers to in-context learn more complex function classes -- namely sparse linear functions, two-layer neural networks, and decision trees -- with performance that matches or exceeds task-specific learning algorithms."
See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066
GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching arithmetic as an algorithm to be performed and taught similarly to how it's done with people, you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after.