News for hackers. Stuff that matters.
Re:Llama 2 – Meta AI by CSMastermind on Tuesday July 18, 2023 @02:49PM (#36777450)
There's a few prompts that I use with every model to compare them. One of the simplest ones is:

> When does the bowl of the winds get used in the wheel of time books?

LLaMA2 fails pretty hard:

> The Bowl of the Winds is a significant artifact in the Wheel of Time series by Robert Jordan. It is first introduced in the third book, "The Dragon Reborn," and plays a crucial role in the series throughout the rest of the books. The Bowl of the Wines is a powerful tool that can control the winds and is used by the Aes Sedai to travel long distances and to escape danger. It is used by the male Aes Sedai to channel the True Power and to perform various feats of magic.

For what it's worth Bard is the only model that I've seen get this question correct with most others hallucinating terrible answers. I'm not sure what it is about this question that trips LLMs up so much but they produce notably bad results when prompted with it.

> Please write a function in JavaScript that takes in a string as input and returns true if it contains a valid roman numeral and false otherwise.

Is another test that I like, which so far no LLM I've tested passes but GPT-4 comes very close.

Here LLaMA2 also fails pretty hard, though I thought this follow up response was pretty funny:

> The function would return true for 'IIIIII' because it contains the Roman numeral 'IV'.

Re:Llama 2 – Meta AI by andsoitis on Tuesday July 18, 2023 @03:00PM (#36777597)
> get this question correct

I am willing to bet a million dollars that it is unlikely any single model will ever be able to answer any question correctly.

The implications then are that one cannot use a single question evaluate whether a model is useful or not.

Re:Llama 2 – Meta AI by nvy on Tuesday July 18, 2023 @03:35PM (#36778063)
>any question

Do you mean "every question"? Because ChatGPT has already answered some of my questions correctly, so if you mean "any" as in "any one of the infinite set of questions" I'll take that bet.

Re:Llama 2 – Meta AI by xsmasher on Tuesday July 18, 2023 @03:27PM (#36777958)
"I don't know" is more correct than making up an answer.
Re:Llama 2 – Meta AI by koonsolo on Tuesday July 18, 2023 @04:04PM (#36778363)
With ChatGPT I sometimes prompt "also indicate how certain you are that your answer is correct". Works pretty good actually.
Re:Llama 2 – Meta AI by nomel on Tuesday July 18, 2023 @04:12PM (#36778458)
I've had very good luck with a follow up "Is that answer correct?"
Re:Llama 2 – Meta AI by sebzim4500 on Tuesday July 18, 2023 @04:23PM (#36778580)
That's not the training objective though. It's like doing exams in school, there is no reason to admit you don't know so you might as well guess in the hopes of a few marks.
Re:Llama 2 – Meta AI by jacquesm on Tuesday July 18, 2023 @04:27PM (#36778624)
If so then that means the training objective is wrong because admitting you do not know something is much more a hallmark of intelligence than any attempt to 'hallucinate' (I don't like that word, I prefer 'make up') an answer.
Re:Llama 2 – Meta AI by og_kalu on Tuesday July 18, 2023 @04:57PM (#36778974)
I guess the brains objective is wrong then seeing how much it's willing to fabricate sense data, memories and rationales when convenient
Re:Llama 2 – Meta AI by jacquesm on Tuesday July 18, 2023 @06:03PM (#36779688)
The brain wasn't designed.
Re:Llama 2 – Meta AI by og_kalu on Tuesday July 18, 2023 @07:03PM (#36780198)
The brain is the result of maximizing biological objective functions.

Since that led to something that fabricates a lot of things very often, saying the objective function of an LLM is "wrong" because it also fabricates is nonsensical.

Re:Llama 2 – Meta AI by jacquesm on Tuesday July 18, 2023 @07:40PM (#36780441)
> The brain is the result of maximizing biological objective functions.

That's not how evolution works at all.

Re:Llama 2 – Meta AI by og_kalu on Tuesday July 18, 2023 @07:50PM (#36780511)
a mutation happens and if that mutation succeeds in ensuring survival, it stays and then spreads. Reproduce is a function evolution maximizes for. Not intentionally sure but that's irrelevant. The whole point of artificial neural networks is that they teach themselves. They get an answer wrong, numbers shift and if those numbers help the next instance they stay or shift as needed. There's no intentionality in the shifting numbers either.
Re:Llama 2 – Meta AI by jacquesm on Tuesday July 18, 2023 @09:30PM (#36781148)
Evolution is not a mechanism that maximizes it is a set of interrelated elements that operate at entirely different levels (molecular, cellular, individual and species) to pass on those traits from one generation to another that result in (possible) change of future individuals within a population, which may affect the survival of those individuals increasing or decreasing the chances of passing those traits on to their (viable!) offspring. It does nothing to ensure survival of any particular individual, at best it may help a trait that has a function in survival to be passed on.

Mutations don't 'succeed' they get passed on or they don't, usually without an immediate effect on the individual, especially not in the individual where the mutation first takes place. But over a longer period mutations may result in a statistical advantage against some environmental factor (including predators) resulting in an increased chance of that particular mutation becoming more widespread. The result is possibly that that mutation ends up being carried by all individuals of the species but that takes a relatively long time and rarely results in 100% coverage in the first generations if at all.

ANN's do not necessarily 'teach themselves', there are supervised ways of using ANNs and there are unsupervised ways of using them.

> They get an answer wrong, numbers shift and if those numbers help the next instance they stay or shift as needed.

No, they optimize for the statistical likelihood of getting all of the answers right across the whole training set, and then we check if it worked by verification with held back data. So it is very well possible that a change in the weights negatively affects some outputs but positively affects others, if the change is a 'net positive' we keep it. To make matters more interesting every now and then we throw away all but a fraction of all the weights.

In the case of supervised learning we (humans) label the data and are the 'teachers' and in the case of unsupervised learning the ANN does its own labeling of clusters of related data (or, more accurately, clusters of data that seem to be related).

There are also intermediary versions where humans do some of the labeling to set up the initial clusters and then the mechanical labeling takes over to rapidly increase the size of the training set taking into account the possibility of getting some of the training data wrong but still coming out ahead across the board.

So yes, in ANNs there is a global maximization around a goal, and we set it up that way. Evolution doesn't have a particular goal, it is a result of a number of interrelated factors not an input and not something that was put together with a goal in mind.

Re:Llama 2 – Meta AI by og_kalu on Tuesday July 18, 2023 @10:59PM (#36781613)
Whether the brain is "designed" or not is really irrelevant to the point here. It has nothing to do with whether an objective function is "wrong" or not.
Re:Llama 2 – Meta AI by jacquesm on Tuesday July 18, 2023 @11:25PM (#36781779)
But it does have to do with whether there is an objective function or not. And there isn't. Brains are the way they are because they evolved that way, because circumstances at some point favored primates with larger brains. Maybe because it allowed us to cooperate, maybe because it enabled skills such as language or higher order thinking and modeling whatever trait you want to substitute for 'the' advantage that allowed our brains to become so large we are in trouble just from being born. Or maybe it wasn't any of that and it was a whole series of small things too tiny to notice individually but with a large enough cumulative effect. None of it had a goal, none of it fit any particular objective function, it's just random chance and local advantages. If natural selection was the whole story that would make the study of evolution of 30 minute thing. But it is far more complex than that. You'd have to explain co-adaptation between two or more species, horizontal gene transfer, the incredibly wasteful ways in which evolution sometimes works. And none of that fits with the notion of an objective function. It's a Platonic ideal. But evolution is incredibly messy and random, with an almost equal chance of going backwards as it has of going forwards on whatever trait you wish to observe. And with the bulk of the mutations resulting in no effect, a negative effect or sterility or death.

The whole notion of 'objective functions' doesn't enter into it, let alone whether or not they are right or wrong. You can retroactively infer one and say that's what evolution is optimizing for but that's confusing cause and effect.

It's about as sophisticated a view of evolution as a billiard ball simulation of the universe. It just doesn't work that way, it's way too simple a representation to have a chance of modeling the observed complexity. You can try to collapse it into such a simplistic model if you want to explain evolution to a small child. Like a placeholder for something better when they're more equipped to deal with the added complexity. Like when we say electrons move from plus to minus and how the simple semiconductor model allows you to design functional circuitry. But the underlying physics is a lot more complex than that.

Re:Llama 2 – Meta AI by og_kalu on Wednesday July 19, 2023 @12:18AM (#36782072)
If intelligence in humans can allow for such behaviour then the same can be said for machines.

It's not suddenly un-intelligent because it faces issues people also face neither is the driving function "wrong".

Sense data prediction and fabrication isn't some trivial side note thing either. It's an essential part of how we process the world.

Re:Llama 2 – Meta AI by jacquesm on Wednesday July 19, 2023 @05:10AM (#36783761)
> If intelligence in humans can allow for such behaviour then the same can be said for machines.

No. This really does not follow. You may explain things to yourself like this but it just isn't true, again. Submarines don't 'swim'. Airplanes do not fly like birds do. Machine intelligence is very much unlike how human intelligence seems to work.

> It's not suddenly un-intelligent because it faces issues people also face neither is the driving function "wrong".

You are seeing something called 'emergent behavior' and are assigning all kinds of properties to the underlying mechanisms that they do not necessarily have.

> Sense data prediction and fabrication isn't some trivial side note thing either. It's an essential part of how we process the world.

So? Think of it as an optimization: if sensors fail then the brain doesn't have the option to throw an error and exit or reboot. So it does the next best thing: it models what the sensor probably would be doing, and hopes it gets it right. This is beneficial and a huge improvement over 'no input'. Such hallucinations of input have absolutely nothing to do with the hallucinations of machine learning software.

Our bodies are by necessity (physical limitations of size, intelligence and the absolutely overwhelming flow of data from our sensory system) going to optimize and condense data so it can be used for reasoning. The way we build our 'world model' is by definition faulty and will never match reality 1:1. But it doesn't have to for it to be very useful. If your brain had to consciously process you vision or hearing data stream it would be absolutely unable to do anything at all. Preprocessing the input, including correcting for partial and complete sensor failure is a very important part of that optimization process. Every organism that has a complex nervous system does some of this, it has nothing to do with us per-se but is simply a feature of how nervous systems evolved, and sets the stage for higher order brain functions.

Re:Llama 2 – Meta AI by clarge1120 on Wednesday July 19, 2023 @03:12AM (#36783016)
…says the atheist, by faith.
Re:Llama 2 – Meta AI by jacquesm on Wednesday July 19, 2023 @05:12AM (#36783773)
This is just trolling for some kind of religious flamewar, even the pope supports evolution so can we please avoid this? Thank you.
Re:Llama 2 – Meta AI by aidenn0 on Tuesday July 18, 2023 @05:05PM (#36779093)
There are plenty of exams that give either partial credit for "I don't know" or negative credit for a wrong answer (to discourage guessing).

Training on internet comments is going to make "I don't know" rather unlikely because when someone asks a question in an online forum and I don't know the answer, people will just not respond rather than responding "I don't know"

Re:Llama 2 – Meta AI by jacquesm on Tuesday July 18, 2023 @04:26PM (#36778612)
Indeed.
Re:Llama 2 – Meta AI by smilliken on Tuesday July 18, 2023 @03:08PM (#36777717)
Of course that has to be the case otherwise you have a halting oracle. It's fitting this was proven by the namesake of the Turing Test.
Re:Llama 2 – Meta AI by b33j0r on Tuesday July 18, 2023 @04:09PM (#36778411)
I got that question wrong, I still have no idea what the correct answer would be. That is extremely obscure.

Any intelligence or simulation might try to guess at an answer to that third-level-of-hell interrogation.

“Why was Spartacus filmed in California near pizza noodle centurions?”

Re:Llama 2 – Meta AI by jacquesm on Tuesday July 18, 2023 @04:25PM (#36778610)
You could of course also answer 'I don't know' which to me is a correct answer, far more so than something you made up.
Re:Llama 2 – Meta AI by b33j0r on Thursday July 20, 2023 @04:21PM (#36806186)
That would make it a more reasonable human. But it’s actually a compendium of everything you and I and shakespeare and the january 6th viking have penned.

The creativity, which we call hallucination, is the advantage of the approach.

If I wanted a search engine for actionable facts, they have worked pretty well for 30 years.

Re:Llama 2 – Meta AI by atonse on Tuesday July 18, 2023 @05:46PM (#36779533)
I'd struggle to find any humans that understand that question without going "huh?"

I've read it 3-4 times and it still doesn't make sense towards the end. So why would we expect these models to make sense of them?

Re:Llama 2 – Meta AI by yodsanklai on Tuesday July 18, 2023 @05:14PM (#36779192)
I would go even further, use models to answer questions only if you don't care whether the answer is correct or not.
Re:Llama 2 – Meta AI by andsoitis on Tuesday July 18, 2023 @11:40PM (#36781869)
what is the use case for that approach?
Re:Llama 2 – Meta AI by yodsanklai on Wednesday July 19, 2023 @05:51AM (#36784019)
any answer that you can check easily: generated code that you can test, text summary/rephrasing. Or questions for which answer aren't critical/objective ("how to procrastinate less").
Re:Llama 2 – Meta AI by pmarreck on Tuesday July 18, 2023 @03:32PM (#36778021)
> Please write a function in JavaScript that takes in a string as input and returns true if it contains a valid roman numeral and false otherwise.

Your question actually isn't worded concisely enough. You don't specify whether the string can merely contain the roman numeral (plus other, non-roman-numeral text), or must entirely consist of just the roman numeral. The way "if it contains" is used colloquially, could imply either.

I'd use either "if it IS a roman numeral" if it must consist only of a roman numeral, and "if there exists a roman numeral as part of the string" or some such, otherwise.

Re:Llama 2 – Meta AI by burkaman on Tuesday July 18, 2023 @03:45PM (#36778172)
I think that makes it a better test. An ideal model would recognize the ambiguity and either tell you what assumption it's making or ask a followup question.
Re:Llama 2 – Meta AI by jsight on Tuesday July 18, 2023 @03:53PM (#36778269)
While that is true, I'm not aware of any model that has been trained to do that. And all models can do is to do what they were trained to do.
Re:Llama 2 – Meta AI by burkaman on Tuesday July 18, 2023 @04:04PM (#36778364)
They are just trained to generate a response that looks right, so they are perfectly capable of asking clarifying questions. You can try "What's the population of Springfield?" for an example.
Re:Llama 2 – Meta AI by Matrixik on Tuesday July 18, 2023 @06:56PM (#36780139)
It's not model but working on top of it: https://www.phind.com/ It's asking clarifying questions.
Re:Llama 2 – Meta AI by jsight on Wednesday July 19, 2023 @04:33PM (#36792825)
That's actually pretty good. Thanks!
Re:Llama 2 – Meta AI by pmarreck on Tuesday July 18, 2023 @03:45PM (#36778177)
That's a great point.
Re:Llama 2 – Meta AI by nomel on Tuesday July 18, 2023 @04:23PM (#36778582)
You can tease this out pretty easily by having it ask question before continuing. My attempt addresses the ambiguity as the first question:

https://chat.openai.com/share/462a7f62-6305-4e2a-a9ae-5f86a6...

I'll often do this, along with "Are there any mistakes in the above?" or something like "Did this fully address the problem statement?"

Re:Llama 2 – Meta AI by renewiltord on Tuesday July 18, 2023 @03:14PM (#36777790)
Interesting. Chat-GPT 4 has the following answer https://chat.openai.com/share/94ef3004-f944-4fc6-bbe5-597d54...

> When does the bowl of the winds get used in the wheel of time books?

> Mention the events leading up to it, the book number and name, and the chapter title if one exists.

> Response: In Robert Jordan's Wheel of Time series, the Bowl of the Winds is used in the 8th book, "The Path of Daggers." The bowl is used as a device to reverse a long-lasting, unnatural heatwave and restore the natural weather patterns.

I think that's right, actually, but it's been years (decades?) since I read that book.

Re:Llama 2 – Meta AI by CSMastermind on Tuesday July 18, 2023 @03:47PM (#36778198)
Oh that's interesting, the last time I checked GPT-4 I got a much worse result.

Yes that answer looks correct to me though it leaves off some later uses in the book series (which is what makes this a slightly tricky question to answer). But in terms of first use it nails it.

Did you add the "Mention the events leading up to it, the book number and name, and the chapter title if one exists." part to the prompt?

I'm wondering if that produces noticeably better results.

Edit:

For reference this is the response I get from Bard:

In A Crown of Swords (Chapter 13) Elayne and Nynaeve first learn about the Bowl of the Winds and begin their search for it.

The Bowl of the Winds is then used in the Wheel of Time series in the following books:

The Path of Daggers (Chapters 5, 24, and 35) - Nynaeve, Talaan, Aviendha, Elayne, Metarra, Garenia, Rainyn, Kirstian, Reanne, Tebreille, Naime, Rysael use the Bowl of the Winds.

Winter's Heart (Chapters 24 and 37) - The Bowl of the Winds is used to stop a massive storm that is threatening to destroy the city of Ebou Dar.

The Gathering Storm (Chapter 34) - The Bowl of the Winds is used to create a powerful windstorm that helps to defeat the Seanchan army at the Battle of Maradon.

A Memory of Light (Chapters 19 and 35) - The Bowl of the Winds is used to fight the weather-controlling abilities of the Dark One's forces during the Last Battle.

Re:Llama 2 – Meta AI by cevn on Tuesday July 18, 2023 @04:07PM (#36778392)
This sounds pretty good according to my memory. I did think it was first mentioned earlier than Path of Daggers. I don't remember it being used in The Last Battle but that was a pretty long chapter ...
Re:Llama 2 – Meta AI by lucubratory on Tuesday July 18, 2023 @07:45PM (#36780469)
It was used in The Last Battle throughout, but never focused on as a main set piece. It was just mentioned peripherally a couple of times that there was a large circle using the Bowl of the Winds to prevent catastrophic weather events from killing everyone during the battle/s.
Re:Llama 2 – Meta AI by cevn on Tuesday July 18, 2023 @09:36PM (#36781179)
Sounds about right now that you mention it. Time for a re read I guess.. :)
Re:Llama 2 – Meta AI by lucubratory on Tuesday July 18, 2023 @09:51PM (#36781262)
As always :P me and my husband will be starting our first WoT reread together (I got him into the books after he loved the show), just as soon as we finish the current series we're on, which is Children of Time by Adrian Tchaikovsky. Absolutely fantastic series.
Re:Llama 2 – Meta AI by cevn on Wednesday July 19, 2023 @02:47PM (#36791333)
Hm... might have to check that out as well, I love sci fi too.. Hopefully he likes the books more than the show, hah.
Re:Llama 2 – Meta AI by lucubratory on Wednesday July 19, 2023 @10:08PM (#36796111)
Children of Time is incredible, I would consider it the best sci-fi novel I've ever read in a similar way to how I consider Wheel of Time the best fantasy series I've ever read.

As for books vs show, in my opinion it's way too early to tell. Both me and him agree that the first season was significantly better than Eye Of The World, but because the later books are much better than EOTW I think the show will need to increase in quality significantly to be better than those later books - I'm excited for season 2, but in season 3 they're tackling the story of The Shadow Rising, which is probably my favourite book in the series. It's an open question in my mind if they can improve quality and characterisation enough to match that bar, although either way I'm going to enjoy it. And more importantly, I think it's not as important to judge them on a season vs book basis, because the reality is the show can't really be judged fairly against a completed book series until it is itself a completed television series. Once they're both finished (aka once I'm an old granny), I expect I'll have a much clearer idea of which rendition of the story I ultimately prefer. I know I'll definitely be talking about it with my husband a lot lol.

Re:Llama 2 – Meta AI by renewiltord on Tuesday July 18, 2023 @04:47PM (#36778851)
I did add that. In general, LLMs do better with some chain of thought prompting. "Let's think step by step" etc
Re:Llama 2 – Meta AI by lucubratory on Tuesday July 18, 2023 @07:49PM (#36780509)
That is an incredibly accurate answer for a niche question (about the best fantasy series of all time, but still less well known than Lord of the Rings so, niche). How is Bard getting that so accurate? My first thought is that maybe they've had an LLM go over their books training corpus and pre-summarise every single book or series, then trained on those summaries as well. Or maybe they did some semi-automated Q/A generation using knowledge extracted from books using traditional techniques (vector search, instance count, first mentioned, etc) to evaluate answers to the model's responses on books.

That's actually really fascinating, I've consistently found that Bard underperforms GPT-4 but this is a resounding win. I wonder what makes this question so different.

Re:Llama 2 – Meta AI by renewiltord on Tuesday July 18, 2023 @11:18PM (#36781733)
It's wild! I'm sure they used the Google Books dataset
Re:Llama 2 – Meta AI by mkl on Tuesday July 18, 2023 @04:45PM (#36778832)
> Here LLaMA2 also fails pretty hard, though I thought this follow up response was pretty funny:

> > The function would return true for 'IIIIII' because it contains the Roman numeral 'IV'.

That's arguably correct. 'IIII' is a valid Roman numeral representation of 4 [1], and the string 'IIIIII' does contain 'IIII'.

[1] https://en.wikipedia.org/wiki/Roman_numerals#Other_additive_...

Re:Llama 2 – Meta AI by sltkr on Tuesday July 18, 2023 @04:56PM (#36778948)
Since you're being pedantic my reply is going to be equally pedantic: no, this is not correct if you understand the difference between numerals and numbers.

A numeral is a written way of denoting a number. So while the string "IIIIIIII..." arguably contains a Roman numeral denoting the number 4 as a substring (if you accept "IIII" as a Roman numeral), it still does not contain the Roman numeral "IV" as a substring.

Or phrased differently, by your logic you might as well say that "IIIIIIII..." contains the Arabic numeral "4". It doesn't.

Re:Llama 2 – Meta AI by nine_k on Tuesday July 18, 2023 @04:57PM (#36778964)
I suppose that current LLMs are incapable of answering such questions by saying "I don't know". The have no notion of facts, or any other epistemic categories.

They work basically by inventing a plausible-sounding continuation of a dialog, based on an extensive learning set. They will always find a plausible-sounding answer to a plausible-sounding question: so much learning material correlates to that.

Before epistemology is introduced explicitly into their architecture, language models will remain literary devices, so to say, unable to tell "truth" from "fiction". All they learn is basically "fiction", without a way to compare to any "facts", or the notion of "facts" or "logic".

Re:Llama 2 – Meta AI by sebzim4500 on Tuesday July 18, 2023 @05:32PM (#36779363)
They kind of do, since the predictions are well calibrated before they go through RLHF, so inside the model activations there is some notion of confidence.

Even with a RLHF model, you can say "is that correct?" and after an incorrect statement it is far more likely to correct itself than after a correct statement.

Re:Llama 2 – Meta AI by lucubratory on Tuesday July 18, 2023 @07:38PM (#36780426)
No, that's a common misconception. They do what they are asked to do, and when they are asked to provide an answer they will provide an answer. If you ask them to provide an answer if they know, or tell you that they don't know if they don't know, they will comply with that quite well, and you'll hear a lot of "I don't know"s for questions it doesn't know the answer to.
Re:Llama 2 – Meta AI by poyu on Tuesday July 18, 2023 @10:48PM (#36781558)
I think the truth is somewhere in between, since I’ve seen both responses: “I don’t know” and something completely made up that was presented as facts.
Re:Llama 2 – Meta AI by sanxiyn on Tuesday July 18, 2023 @10:02PM (#36781317)
In my experience, GPT-4 answers "I don't know" fairly frequently.
Re:Llama 2 – Meta AI by 8n4vidtmkvmk on Tuesday July 18, 2023 @05:35PM (#36779406)
Contains a valid roman numeral or is a valid roman numeral? My first instinct was it should return true if the string contains V or I or M or... Whatever the other letters are.
Re:Llama 2 – Meta AI by ISV_Damocles on Tuesday July 25, 2023 @01:16PM (#36865778)
So this comment inspired me to write a Roman Numeral to Integer function in out LLM-based programming language, Marsha: https://github.com/alantech/marsha/blob/main/examples/genera...