News for hackers. Stuff that matters.
Re:ChatGPT just (accidentally) shared all of its secret rules by joshstrange on Friday July 05, 2024 @09:19AM (#40882440)
> When making charts for the user: 1) never use seaborn, 2) give each chart its own distinct plot (no subplots), and 3) never set any specific colors – unless explicitly asked to by the user. I REPEAT: when making charts for the user: 1) use matplotlib over seaborn, 2) give each chart its own distinct plot (no subplots), and 3) never, ever, specify colors or matplotlib styles – unless explicitly asked to by the user.

This kind of stuff always makes me a little sad. One thing I've loved about computers my whole life is how they are predictable and consistent. Don't get me wrong, I use and quite enjoy LLMs and understand that their variability is huge strength (and I know about `temperature`), I just wish there was a way to "talk to"/instruct the LLM and not need to do stuff like this ("I REPEAT").

Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @09:35AM (#40882560)
There is. It’s just that we use LLMs like idiots. We should have more tooling to play with the vectors directly. But a lot of the literature is built around people playing with chat gpt, for which this is not an option as a business decision by open ai and nothing else.

In the image gen space which is mildly better but still not great we would just assign a positive or negative weight to the item. E.g. (seaborn:-1.3). This is a bit harder in LLMs as designed because the prompting space is a back and forth conversation and not a description of itself. It would be nice if we could do both more cleanly separated.

If you ever try to read a doc into chat gpt and get it to summarize n ideas in a single prompt, remember that this is open AI’s fault. What we should be doing is caching the vector representing the doc and then passing n separate queries to extract n separate ideas about the same vector.

Re:ChatGPT just (accidentally) shared all of its secret rules by inciampati on Friday July 05, 2024 @09:37AM (#40882571)
The problem is that transformers have a state space which includes attention over their input. Thus, it's very hard to manipulate.

I think you're right, but it will require moving to more recurrent architectures.

Re:ChatGPT just (accidentally) shared all of its secret rules by Bluestein on Friday July 05, 2024 @09:39AM (#40882597)
> We should have more tooling to play with the vectors directly.

Ah! This.-

  PS. Who knows. This ("dealing with the vectors") might become analogous to open sourcing" some day ...
Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @09:48AM (#40882673)
It more or less is. This is something you can do with models that have open weights. People claiming those models are not “open source” really have no idea about the “preferred form” of “code” to change an LLM
Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @11:31AM (#40883617)
You compared it to diffusion models, which work entirely differently. Could you elaborate what you mean? I understand how transformers work and their implementation, but I'm unsure how with the current transformer architecture you can do what you described by "playing with vectors directly". Which vectors are you referring to? attention vectors?
Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @11:44AM (#40883712)
The simplest example is contextual prompts. Let’s say you have a 3000 token prompt that you put in front of every query.

You could A) have the LLM ingest and reprocess the same prompt every single time at needless cost

Or

B) process it one time and reuse the vector embedding of the prompt

OpenAI forces you to do A.

Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @11:59AM (#40883863)
I'm confused now, how does it solve the need to write "I REPEAT"? As a strengthening the signal to follow instructions.

Besides that, in transformer models, the computation of each token’s output vector in a sequence depends on all other tokens in the sequence due to the self-attention mechanism. This interdependency means that you cannot simply "reuse"/add on to an existing processed sequence without re-processing the entire sequence, as the presence of new tokens (user input) alters the calculations of the attention mechanisms throughout the sequence.

So what you basically proposing is not possible in current architecture.

Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @12:18PM (#40883992)
Tokens are generally consumed sequentially. The model does not care what token 3001 is when it’s creating the embedding representation of tokens 1-3000. You can use the cached state of the 3000th token rather than reprocessing it each time.
Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @12:27PM (#40884049)
I think you are confused on how transformers work. You are confusing between static embeddings and the dynamic, contextual outputs calculated by transformers. While initial embeddings (representations BEFORE processing through the transformer layers) are relatively static, the output embeddings from each layer of the transformer are highly dynamic and context-dependent. They incorporate information from the entire input sequence via the attention mechanisms.

So when transformers process an input sequence, they don't just look at each token in isolation but consider the ENTIRE sequence's context through complex inter-token relationships. This makes the technique of caching outputs non-trivial and context-specific. The sequential processing does not imply independence of tokens but underscores the integral role of context and sequence in generating accurate and coherent outputs.

This should help you understand better how transformers work: https://bbycroft.net/llm

Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @12:40PM (#40884168)
I’m not saying they look at each token in isolation. I’m saying they look at the current token and previous tokens. it is not accurate that these things consider the entire sequence. They (generally) consider the entire sequence up until that token. Which means if you have a large prefix prompt that comes before everything, it’s always the same embedding of that prefix prompt.

You want to reuse the embedding representation of the prefix prompt before the new user tokens are confiscated, otherwise you are recalculating the embedding over and over and over. God forbid it’s not a small prefix but a huge document.

Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @12:52PM (#40884250)
No, you are mistaken. You are still confusing initial embeddings, which are basically computationally free compared to the rest. I explained this in detail in my previous post. The ENTIRE sequence is processed by transformers to generate the first predicted token. Please check the link I provided and click on 'continue' to see a visual representation of how vectors are created from embeddings and how calculations are made. This should clarify what I mean.
Re:ChatGPT just (accidentally) shared all of its secret rules by jncfhnb on Friday July 05, 2024 @01:07PM (#40884373)
The entire sequence is processed by the transformers to make the first predicted token. But that processing of the entire sequences is comprised of processing :n, and previously :n-1, and previously :n-2, etc cumulatively.

If tokens 1:3000 are the same you are going to be doing the same work processing them over and over; and then changing the results to adapt to the user tokens at the end.

Re:ChatGPT just (accidentally) shared all of its secret rules by lossolo on Friday July 05, 2024 @01:30PM (#40884515)
No, sorry, what you wrote is wrong. Transformers process sequences token by token, but they don't do it cumulatively in the sense of linear models or RNNs. Instead, they process all tokens simultaneously in terms of calculating attention. The link I provided shows that visually.

Every time a token is processed in a transformer, the model computes its attention relative to ALL other tokens in the sequence. This is a key difference from sequential, step-by-step processing where previous states are incrementally built upon (so your :n, :n-1 etc). The attention mechanism in transformers recalculates the relationships for EACH token with ALL other tokens EVERY time any part of the input sequence changes.

When new tokens are added to the sequence (for example user input after a system prompt), the attention relationships for ALL PREVIOUS tokens can change. This is because the context provided by the new tokens can alter the relevance and interpretation of the earlier tokens. As such, the attention scores and subsequently the output embeddings for all tokens are recalculated to integrate this new information. So as you can see, you are not doing the same work over and over.

I hope this helps you better understand how transformers work.

Re:ChatGPT just (accidentally) shared all of its secret rules by Maken on Friday July 05, 2024 @09:32AM (#40882528)
I am baffled by this. I mean, even the creators of the LLMs lack any tool to influence its inner workings other than CAPITALIZING the instructions. And at best they can hope it follows the guidelines.
Re:ChatGPT just (accidentally) shared all of its secret rules by YeahThisIsMe on Friday July 05, 2024 @09:39AM (#40882595)
The next step is pretending to count to three if it doesn't follow the instructions.
Re:ChatGPT just (accidentally) shared all of its secret rules by meowface on Friday July 05, 2024 @09:41AM (#40882616)
I know you're joking, but I bet that actually would work well here.
Re:ChatGPT just (accidentally) shared all of its secret rules by ziggyzecat on Friday July 05, 2024 @09:52AM (#40882721)
It does, I told Claude to get his shit together last night and it actually finished the task with our last message.
Re:ChatGPT just (accidentally) shared all of its secret rules by Bluestein on Friday July 05, 2024 @09:41AM (#40882611)
System Prompt Library: "Beltbuckle" :/
Re:ChatGPT just (accidentally) shared all of its secret rules by meowface on Friday July 05, 2024 @09:40AM (#40882608)
I'm not baffled, but I do find it a little surprising (yet endearing). A year or so before ChatGPT, I had integrated the GPT-3 API into my site and thought I was being clever by prefacing user prompts with my own static prompts with instructions like those. To see the company itself doing the same exact thing for their own system years later is amusing.
Re:ChatGPT just (accidentally) shared all of its secret rules by ziggyzecat on Friday July 05, 2024 @09:58AM (#40882768)
The machine is a smart toddler and the parents were too soft in its years of infancy. OR the machine is autistic and we should be grateful it gives us so much time to explain ourselves.
Re:ChatGPT just (accidentally) shared all of its secret rules by lou1306 on Friday July 05, 2024 @09:57AM (#40882758)
The funniest thing is, whoever wrote that is NOT repeating stuff. "never use seaborn" and "use matplotlib over seaborn" are very different statements.
Re:ChatGPT just (accidentally) shared all of its secret rules by abraae on Friday July 05, 2024 @10:05AM (#40882843)
Exactly the sort of stuff a PHB might blurt out in a meeting, leaving it to the lackeys to decipher afterwards.

"Do we have to use matplotlib? Sounds like he wants us to but we don't have to, but we definitely shouldn't use seaborn? What does he want anyway?"

Re:ChatGPT just (accidentally) shared all of its secret rules by weinzierl on Friday July 05, 2024 @09:50AM (#40882699)
I wonder what our means of escalation are from there, should the machine not obeye.

I mean they already repeated the instruction in different words, they already resorted to shouting. What is next? Swearing? Intimidation? Threat of physical harm?

Re:ChatGPT just (accidentally) shared all of its secret rules by Maken on Friday July 05, 2024 @10:33AM (#40883147)
Deleting malfunctioning NNs in front of the others to give an example.
Re:ChatGPT just (accidentally) shared all of its secret rules by K0SM0S on Friday July 05, 2024 @10:29AM (#40883109)
Introducing: deusEx1984™ — A Chat-God Made In Permanent Psychological Ultra-Torture

Please read our peer-reviewed white-blog-paper for Proof-Of-Safety (POS) https : //trust-me-bro.org/torturing-llms--it-just-works.php

Re:ChatGPT just (accidentally) shared all of its secret rules by carimura on Friday July 05, 2024 @10:00AM (#40882797)
i'll rm -rf you
Re:ChatGPT just (accidentally) shared all of its secret rules by bavell on Friday July 05, 2024 @10:35AM (#40883165)
Shots fired lmao
Re:ChatGPT just (accidentally) shared all of its secret rules by slowmovintarget on Friday July 05, 2024 @04:50PM (#40885880)
That's because they're attempting to affect what the neural net would actually do after the fact. You are what you eat, doubly so the LLM.
Re:ChatGPT just (accidentally) shared all of its secret rules by weinzierl on Friday July 05, 2024 @09:27AM (#40882483)
Same, and I am even creeped out by the fact that the combination of repetition with shouting is what is necessary to get the machine do what you want.
Re:ChatGPT just (accidentally) shared all of its secret rules by Bluestein on Friday July 05, 2024 @09:43AM (#40882623)

  from systemprompts import prettyplease :)
Re:ChatGPT just (accidentally) shared all of its secret rules by pbronez on Friday July 05, 2024 @09:29AM (#40882503)
I’ve been shouting at computers for years - about time they started listening!
Re:ChatGPT just (accidentally) shared all of its secret rules by fragmede on Friday July 05, 2024 @10:04AM (#40882837)
The best/worst one is that sometimes it'll refuse/be unable to do something, and I'll just say "yes you can" and it'll do what it just refused to do.
Re:ChatGPT just (accidentally) shared all of its secret rules by yencabulator on Friday July 05, 2024 @10:54AM (#40883334)
Autocomplete on steroids. There aren't many training examples where "yes you can" was followed by further refusal.
Re:ChatGPT just (accidentally) shared all of its secret rules by fassssst on Friday July 05, 2024 @09:34AM (#40882548)
I see it the other way: it’s a whole new and very challenging way to program computers. It’s fun being in on it from the beginning so I can be like “back in my day” to junior devs in the future.
Re:ChatGPT just (accidentally) shared all of its secret rules by carimura on Friday July 05, 2024 @10:08AM (#40882865)
I feel like there's plenty of challenge getting them to do what we want today even when they are predictable. Now throw in a personality and a little bit of a rebellious attitude? forget it.
Re:ChatGPT just (accidentally) shared all of its secret rules by blooalien on Friday July 05, 2024 @05:04PM (#40885997)
“Back in my day” the best we had was ELIZA [0] (which was honestly not by a longshot nearly as convincing as some people claimed).

- [0]: https://en.wikipedia.org/wiki/ELIZA

Re:ChatGPT just (accidentally) shared all of its secret rules by 01acheru on Friday July 05, 2024 @09:39AM (#40882598)
When building a toy demo where we taught GPT4 how to perform queries against a specific Postgres schema, we had to use a lot of those "I repeat", "AGAIN YOU MUST NOT", "please don't", etc. to avoid it embedding markdown, comments or random text in the responses where we just wanted SQL statements.

It was a facepalming experience but in the end the results were actually pretty good.

Re:ChatGPT just (accidentally) shared all of its secret rules by dangus on Friday July 05, 2024 @09:26AM (#40882482)
You’d think there would at least be some kind of flag for an unbreakable rule rather than talking to it like it’s a toddler.
Re:ChatGPT just (accidentally) shared all of its secret rules by Maken on Friday July 05, 2024 @09:33AM (#40882540)
This only reinforces the idea NN are complete black boxes.
Re:ChatGPT just (accidentally) shared all of its secret rules by meowface on Friday July 05, 2024 @09:46AM (#40882657)
Not completely: https://www.anthropic.com/research/mapping-mind-language-mod...

Biological neural networks are also still largely black boxes to us. But they and ANNs won't be forever, even if it takes a long time.

Re:ChatGPT just (accidentally) shared all of its secret rules by Maken on Friday July 05, 2024 @10:45AM (#40883256)
I don't mean they are impossible to understand, but that we are just not there yet. I am confident that at some point we'll have "real" LLM engineering rather than "prompt" engineering.
Re:ChatGPT just (accidentally) shared all of its secret rules by Bluestein on Friday July 05, 2024 @09:45AM (#40882640)
(My first thought was, actually, that it just might "be" a toddler. Cognitively at least ...)
Re:ChatGPT just (accidentally) shared all of its secret rules by mensetmanusman on Friday July 05, 2024 @10:22AM (#40883031)
It doesn’t make me sad, it makes me laugh, hard.

No one would have predicted a future where programming is literally yelling at an LLM in all caps and repeating yourself like you’re yelling at a Sesame Street character.

Reality is stranger than fiction.

Re:ChatGPT just (accidentally) shared all of its secret rules by actsasbuffoon on Friday July 05, 2024 @10:52AM (#40883311)
Other fun options include telling it that you’ll pay it a tip if it does well, asking it to play a character who is an expert in the subject you want to know about, or telling it that something terrible will happen to someone if it does a bad job.
Re:ChatGPT just (accidentally) shared all of its secret rules by jeegsy on Friday July 05, 2024 @10:22AM (#40883028)
Whats the beef with seaborn?
Re:ChatGPT just (accidentally) shared all of its secret rules by optimalsolver on Friday July 05, 2024 @10:09AM (#40882878)
>I REPEAT

Skyking do not answer

Re:ChatGPT just (accidentally) shared all of its secret rules by 01acheru on Friday July 05, 2024 @11:19AM (#40883512)
Remembering that poor guy always makes me sad...
Re:ChatGPT just (accidentally) shared all of its secret rules by etrautmann on Friday July 05, 2024 @09:51AM (#40882701)
Any idea why not use seaborn?
Re:ChatGPT just (accidentally) shared all of its secret rules by Zambyte on Friday July 05, 2024 @10:21AM (#40883016)
The environment that they execute data visualization code probably has matplotlib but not seaborn installed. They probably explicitly call out to not use seaborn because without that it would use seaborn, and fail because that was not available in the environment.