The variance he is seeing in the output is primarily the product of random chance, rather than changes in the model. Specifically this "unicorn" that he found today is likely just random chance and there was no changes in the model between yesterday and today that lead to it arising.
If he wanted to track changes in the model for real, he would have to ask multiple questions per day and try to infer some type of distribution characterization and then see if that changes over time. That is much more complex and not what he is doing.
This is just a curious experiment that doesn't mean much.
It is a fun experiment though.
Perhaps in a few months half of the generated pictures will look like unicorns, perhaps in more months they will all be unicorns but 2 or 3 will look way more detailed instead of drawn by a 4 year old, etc. We just need to wait longer for the signal to break through the noise.
> We just need to wait longer for the signal to break through the noise.
Currently, what we are observing is primarily noise with very little signal.
The difference over months or years is what is interesting.
SVG can present arbitrarily complex graphics. The underlying display tech supports what ever fidelity GPT will eventually mature into.
Has GPT plateaud? Will it be stuck forever at this hilariously naive level of competence at SVG art? Will it mature into Midjourney level competence? I have no frigging clue. Since the token context is so small I imagine it will put limitations to the complexity of SVG art piece.
But I don't know. And it's fun to have a daily measure.
As a software engineer with a penchant for graphics asking GPT to draw complex graphic shapes was one of the first tests I did for it. It's extremely interesting for me to collect progress data, no matter how noisy.
I have no idea if GPT will ever mature beyond these squiggles but if it does, this track record will have at least considerable artistic value, if nothing else.
Let's think about it:
1. It has to output SVG [1] 2. It is given a text based representation of what it must draw[2] 3. It must then somehow convert words -- the concept of a unicorn: equine with a horn, white, maybe rainbows? -- into SVG code, and attempt to convey both their location, shape, colour, appearance, with code.
And keep in mind, this is just a token predictor. I doubt there is much data in its training that is this specific.
So while it's quite far from science, for me, it's a bit of fun and I get emails every now and then remarking on things like the turd of May (2023-05-18) and it lightens the mood every now and then, which I think ultimately, is worth it.
[1] System: You are a helpful assistant that generates SVG drawings. You respond only with SVG. You do not respond with text.
[2] User: Draw a unicorn in SVG format. Dimensions: 500x500. Respond ONLY with a single SVG string. Do not respond with conversation or codeblocks.
See: https://github.com/adamkdean/gpt-unicorn/blob/master/src/lib...
That's one the reasons why GPT when it works, feels magical. SVG art does not need to be in it's training set, as long as it knows how to present geometric concepts in SVG.
A good unicorn would require capabilities something like "the outline of unicorn is composed of lines {...}." -> "export lines as svg".
"As mentioned in the hacker news discussion, the model doesn't change daily. [...] As OpenAI releases incremental updates, we'll see the model change automatically and be able to judge outputs. A single sample per day leads to quite different results, but that's fine I think. What I expect to see a year from now is an evolution of output. In variance: how varied are the outputs over a month?"
So yes, they could just produce 100 images with each new model release, but chose to spread those out over 1 per day instead. Is it the most scientific way to measure progress? No. Is it more fun and interesting to check back daily? Probably.
[0] https://adamkdean.co.uk/posts/gpt-unicorn-a-daily-exploratio...
That's worse logic. How would you visualize the very large sample you would get? Even with the current 118 samples (one per day) it's already difficult to find a pattern.
Would you "average" the samples?? That would not help IMO, you would need to average the score of each image, which requires either manually doing it or finding a reliable algorithm to do it automatically, but good luck with that.
So, a sample per day which allows clearly visualizing any change in the results over months and years is a valuable thing to do and I find it hard to improve on the methodology. You just need to keep in mind that one single picture from the sample is not enough, no one is going to disagree with that... but that doesn't make it "bad logic" and it's pretty thoughtless to say so.
But maybe you could covert it to a linear monotonic measure?
You could pass it to an image recognition model and see record the degree to which it thinks it is an:
animal horse unicorn
Basically if it fails to be a unicorn, see if it is a horse and if it fails to be a horse check if it is an animal. This gives you some type of linear measure if you place these three measures along the same axis adjacently. Then you can transform each image to a floating point number and then characterize the distribution.
We don't care what another algorithm "thinks". We want to see if what it draws is humanly interpretable as a unicorn.
I appreciate all the comments around determinism, sampling, scientific method, but as I said when I posted this just after building, it really is just for fun and to see, over time, if the general mish mash of outputs become more refined without any changes to the prompt (which doesn't aid it through CoT/ToT or improving on previous attempts etc.)
This is absolutly fine and it should start showing unicorn like drawings over a longer period and potentially finetuned ones over a longer period of time when the model changes.
Huh? What "original post"? This is an experiment, today the model drew something resembling a unicorn. Tomorrow we will see how the experiment goes again. I see no associated analysis, so what makes your "head hurt".
> The idea behind GPT Unicorn is quite simple: every day, GPT-4 will be asked to draw a unicorn in SVG format. This daily interaction with the model will allow us to observe changes in the model over time, as reflected in the output.
I believe he's trying to find an unicorn similar in style to the one generated by the original researcher.
It's so sad that openai has a far more capable model internally that it can't give open access to because of safety (or any other argument).
Parallelism doesn't magically add non-determinism of this kind unless you intentionally build it to be non deterministic. Nothing prevents you from processing an array in order in parallel.
You would have to explicitly order the terms prior to reduction but you don't always have that level of control.
100% correct if you remove processing time from the equation.
In reality, Nvidia Cuda calculations run much faster if you let it schedule the order of floating points operations itself. This makes the ordering different from run to run.
This in turn causes the results to be non-deterministic.
One idea is that the cause is batched inference in sparse MoE (mixture of experts) models.
https://152334h.github.io/blog/non-determinism-in-gpt-4/
HN discussion: https://news.ycombinator.com/item?id=37006224
Currently there are two different GPT4 models represented in the samples, with quite significant quality difference between them. The quality (and variance in quality within a single model!) is interesting to see in such a comparison.
(submitter here) You're correct, it's not the goal of the project. It would be fair to say there is no goal other than to ask GPT to draw a unicorn every day, and through it, create a talking point and potential fun for people who follow along.
Like most of HN.
1. A change in the current model number (eg. gpt-3.5-turbo-0613)
2. On ChatGPT UI, the date at the bottom (eg. August 2023)
So it isn’t correct to say “it is incredibly obvious that nothing has happened”. Not that obvious to me.
A bit like how you can never tell for sure if Coca Cola has tweaked their formula, or McDonalds has changed the recipe for its signature sauce. Only in this case, the model number going up or the date becoming more recent leads credence to something having changed.
Also it's fun to see each daily unicorn.
Even if it's not sparse MoE, chances are high that the non-determinism is introduced somewhere purely as a performance optimization. The article speculates that OpenAI knows this well and hides it to protect the model internals.
[1] https://152334h.github.io/blog/non-determinism-in-gpt-4/
It is well known that the Nvidia parallel processing optimizations cause non-deterministic results.
It's easy to get deterministic results as far as that goes. They've just elected not to do that, since it would run much slower.
You need only to look at the discourse around the Tesla FSD superusers to see this: they report a glitch at an intersection one day, then believe the next day it was "fixed" by the AI.
No, that's the problem. You can't. You should be able to, but you can't. If you could, they wouldn't be scary. But we have Temperature Zero, different results. Because no one gave enough of a shit when coding them, and no one gives enough of a shit to try to fix the issue.
This is what in any other industry would be called gross negligence.
No, you can't. For the latest GPT models and the way they are run, this doesn't work anymore, making the experiment completely illogical. Some of the reasons are explained here pretty well: https://152334h.github.io/blog/non-determinism-in-gpt-4/
I agree that some people take it too far, but most seem to be using metaphor to abstract away the underlying complexities and facilitate conversation. I did the same thing back in college, anthropomorphizing cells and even molecules when I was learning about microbiology and biophysics (e.g., kinesins[1] are a family of postal workers who work hard to deliver packages to their clients in a timely manner, and they spend their free time going for strolls and practicing on their favorite tightropes). I now do it in my day-to-day work at an AI/ML shop to communicate not just what the entire pipeline is doing, but what individuals layers or encoders are doing, or even what variables in an equation are doing. I find that my colleagues and I are better able to understand and remember concepts and ideas when they're communicated as part of an anthropomorphic story, but we scarcely forget that the things we're dealing with aren't human.
But maybe I'm missing the point, and you're really worried about that first, smaller group of people who have really swallowed the Kool aid. To that, I can say that I don't think their behavior exceeds the baseline craziness/weirdness that I've come to expect from humanity. I'm sure there are far more people who believe in astrology than who believe that we've achieved true humanlike artificial general intelligence, for example.
What I find scary is how much trust people put into the answers.
Today, I saw someone saying "Look! ChatGPT can design my home solar electrical circuit". That kind of thing will lead to new Darwin awards being given out.
Thing is, if people will trust it to do that right, when will politicians write policy papers with it? Let's face it, they already are. Other people won't want to read those long papers, so they'll ask it to summarise it for them. At that point you have LLMs writing laws and reviewing laws. It's all fine as long as nobody enforces them.
Right?
It's not particularly "scary" to me that people do that. I remember Boomer and Scooby Doo bots that people anthropomorphized and those were warbots from barely 10 y ago.
I suppose, in today's parlance, "it's scary how much people use fear-oriented language for normal things".
It seems to be the current trend in communication nowadays: A race to induce the strongest reaction possible.
Why should someone be scared of that? We have anthropomorphised chairs, brooms and whatnot since the Walt Disney times and I am sure before in classic literature (I am not that literate to know for certain).
I prefer to be 'amazed' or 'excited' about what is happening: It means that AI is getting to a point where people feel it more 'relatable'. We are getting to that point in our technology development. The number of things we will be able to do with a technology with which we can interact that seamlesly is great.
I doubt it’s trigger warnings because they usually don’t “warn scary things”.
Also is there any data to back up that scary and fear is used more today than in the past? That seems unlikely.
Some people even anthropomorphized Eliza. To your point: if you cherry pick enough and survey enough people; “some people” will do just about anything.