News for hackers. Stuff that matters.
Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @09:35AM (#40882560)
There is. It’s just that we use LLMs like idiots. We should have more tooling to play with the vectors directly. But a lot of the literature is built around people playing with chat gpt, for which this is not an option as a business decision by open ai and nothing else.

In the image gen space which is mildly better but still not great we would just assign a positive or negative weight to the item. E.g. (seaborn:-1.3). This is a bit harder in LLMs as designed because the prompting space is a back and forth conversation and not a description of itself. It would be nice if we could do both more cleanly separated.

If you ever try to read a doc into chat gpt and get it to summarize n ideas in a single prompt, remember that this is open AI’s fault. What we should be doing is caching the vector representing the doc and then passing n separate queries to extract n separate ideas about the same vector.

Re:ChatGPT just (accidentally) shared all of its secret rules by inciampati on Friday July 05, 2024 @09:37AM (#40882571)
The problem is that transformers have a state space which includes attention over their input. Thus, it's very hard to manipulate.

I think you're right, but it will require moving to more recurrent architectures.

Re:ChatGPT just (accidentally) shared all of its secret rules by Bluestein on Friday July 05, 2024 @09:39AM (#40882597)
> We should have more tooling to play with the vectors directly.

Ah! This.-

  PS. Who knows. This ("dealing with the vectors") might become analogous to open sourcing" some day ...
Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @09:48AM (#40882673)
It more or less is. This is something you can do with models that have open weights. People claiming those models are not “open source” really have no idea about the “preferred form” of “code” to change an LLM
Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @11:31AM (#40883617)
You compared it to diffusion models, which work entirely differently. Could you elaborate what you mean? I understand how transformers work and their implementation, but I'm unsure how with the current transformer architecture you can do what you described by "playing with vectors directly". Which vectors are you referring to? attention vectors?
Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @11:44AM (#40883712)
The simplest example is contextual prompts. Let’s say you have a 3000 token prompt that you put in front of every query.

You could A) have the LLM ingest and reprocess the same prompt every single time at needless cost

Or

B) process it one time and reuse the vector embedding of the prompt

OpenAI forces you to do A.

Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @11:59AM (#40883863)
I'm confused now, how does it solve the need to write "I REPEAT"? As a strengthening the signal to follow instructions.

Besides that, in transformer models, the computation of each token’s output vector in a sequence depends on all other tokens in the sequence due to the self-attention mechanism. This interdependency means that you cannot simply "reuse"/add on to an existing processed sequence without re-processing the entire sequence, as the presence of new tokens (user input) alters the calculations of the attention mechanisms throughout the sequence.

So what you basically proposing is not possible in current architecture.

Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @12:18PM (#40883992)
Tokens are generally consumed sequentially. The model does not care what token 3001 is when it’s creating the embedding representation of tokens 1-3000. You can use the cached state of the 3000th token rather than reprocessing it each time.
Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @12:27PM (#40884049)
I think you are confused on how transformers work. You are confusing between static embeddings and the dynamic, contextual outputs calculated by transformers. While initial embeddings (representations BEFORE processing through the transformer layers) are relatively static, the output embeddings from each layer of the transformer are highly dynamic and context-dependent. They incorporate information from the entire input sequence via the attention mechanisms.

So when transformers process an input sequence, they don't just look at each token in isolation but consider the ENTIRE sequence's context through complex inter-token relationships. This makes the technique of caching outputs non-trivial and context-specific. The sequential processing does not imply independence of tokens but underscores the integral role of context and sequence in generating accurate and coherent outputs.

This should help you understand better how transformers work: https://bbycroft.net/llm

Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @12:40PM (#40884168)
I’m not saying they look at each token in isolation. I’m saying they look at the current token and previous tokens. it is not accurate that these things consider the entire sequence. They (generally) consider the entire sequence up until that token. Which means if you have a large prefix prompt that comes before everything, it’s always the same embedding of that prefix prompt.

You want to reuse the embedding representation of the prefix prompt before the new user tokens are confiscated, otherwise you are recalculating the embedding over and over and over. God forbid it’s not a small prefix but a huge document.

Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @12:52PM (#40884250)
No, you are mistaken. You are still confusing initial embeddings, which are basically computationally free compared to the rest. I explained this in detail in my previous post. The ENTIRE sequence is processed by transformers to generate the first predicted token. Please check the link I provided and click on 'continue' to see a visual representation of how vectors are created from embeddings and how calculations are made. This should clarify what I mean.
Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @01:07PM (#40884373)
The entire sequence is processed by the transformers to make the first predicted token. But that processing of the entire sequences is comprised of processing :n, and previously :n-1, and previously :n-2, etc cumulatively.

If tokens 1:3000 are the same you are going to be doing the same work processing them over and over; and then changing the results to adapt to the user tokens at the end.

Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @01:30PM (#40884515)
No, sorry, what you wrote is wrong. Transformers process sequences token by token, but they don't do it cumulatively in the sense of linear models or RNNs. Instead, they process all tokens simultaneously in terms of calculating attention. The link I provided shows that visually.

Every time a token is processed in a transformer, the model computes its attention relative to ALL other tokens in the sequence. This is a key difference from sequential, step-by-step processing where previous states are incrementally built upon (so your :n, :n-1 etc). The attention mechanism in transformers recalculates the relationships for EACH token with ALL other tokens EVERY time any part of the input sequence changes.

When new tokens are added to the sequence (for example user input after a system prompt), the attention relationships for ALL PREVIOUS tokens can change. This is because the context provided by the new tokens can alter the relevance and interpretation of the earlier tokens. As such, the attention scores and subsequently the output embeddings for all tokens are recalculated to integrate this new information. So as you can see, you are not doing the same work over and over.

I hope this helps you better understand how transformers work.