News for hackers. Stuff that matters.
Re:Advanced NLP with SpaCy by yorwba on Saturday September 09, 2023 @06:58AM (#37443995)
1. Prompt:

"""

The following English text contains several French phrases. List all of them.

Text: Gabonese President Ali Bongo Odimba was deposed in a coup d'etat spearheaded by his father's former aide-de-camp Brice Oligui Nguema, shortly after the announcement that Bongo had won the 2023 election.

List of French phrases from the text:

-

"""

Response:

"""

coup d'etat

- aide-de-camp

"""

2. Prompt:

"""

Turn all words in the following text into their lemma form.

Text: Gabonese President Ali Bongo Odimba was deposed in a coup d'etat spearheaded by his father's former aide-de-camp Brice Oligui Nguema, shortly after the announcement that Bongo had won the 2023 election.

Lemmatized text:

"""

Response:

"""

Gabonese President Ali Bongo Odimba be depose in a coup d'etat spearhead by his father's former aide-de-camp Brice Oligui Nguema, shortly after the announcement that Bongo have win the 2023 election.

"""

Maybe too slow or too inaccurate or too expensive compared to bespoke solutions, but certainly not completely useless.

If you want a different tokenization, just turn the LLM response back into text and retokenize it.

Re:Advanced NLP with SpaCy by croes on Saturday September 09, 2023 @11:57AM (#37446829)
LLMs is non deterministic, so tomorrow the same prompt can give you a totally different result.

That's what renders it nearly useless for such tasks, because you always have to check everything.

Re:Advanced NLP with SpaCy by yorwba on Saturday September 09, 2023 @12:35PM (#37447232)
There's nothing about LLMs that is inherently nondeterministic. Sure, if you're using some API you have no control over, anything could happen. But if you run it on your own hardware, you can make it as deterministic as any classic NLP approach.

And whether you always have to check everything is a separate question from nondeterminism. You could have a deterministic heuristic that is often wrong in a domain where mistakes are fatal, or you could have a nondeterministic model that is almost always correct for a task where errors cost next to nothing.

Re:Advanced NLP with SpaCy by squeaky-clean on Sunday September 10, 2023 @04:17AM (#37453890)
ChatGPT 4 is inherently non-deterministic. Even across the same exact model version with temperature set to 0.

https://ai.plainenglish.io/gpt-4-is-non-deterministic-af2028...

Re:Advanced NLP with SpaCy by yorwba on Sunday September 10, 2023 @08:55AM (#37455409)
That's what I meant by "if you're using some API you have no control over, anything could happen." OpenAI could offer access to a fully deterministic version at a higher price point, but they choose not to and there's nothing you can do about that.

FYI, your source is medium-walled blog spam of https://152334h.github.io/blog/non-determinism-in-gpt-4/

Re:Advanced NLP with SpaCy by squeaky-clean on Monday September 11, 2023 @02:06AM (#37463472)
If you'd read the source you linked you'd see that no, the system is inherently non-deterministic. They cannot sell you a deterministic version of GPT-4. There is no such thing. They could sell you a deterministic version of some other inferior model.
Re:Advanced NLP with SpaCy by yorwba on Monday September 11, 2023 @07:15AM (#37465542)
From the source, which I read:

As a consequence, the model is no longer deterministic at the sequence-level, but only at the batch-level

They could sell you a version where you shove in a whole batch, pay $batch_size as much, and always get the same result for the same batch.

Re:Advanced NLP with SpaCy by dartos on Saturday September 09, 2023 @06:01PM (#37450727)
Only the training is non deterministic. As long as you use the same seed for inference, it’ll give you the same answer.

ChatGPT changes this seed under the hood with each request

Re:Advanced NLP with SpaCy by squeaky-clean on Sunday September 10, 2023 @04:18AM (#37453894)
Even if you keep the same seed, it is non-deterministic.

https://ai.plainenglish.io/gpt-4-is-non-deterministic-af2028...

Re:Advanced NLP with SpaCy by dartos on Tuesday September 12, 2023 @07:36PM (#37490159)
That link is dead. Either way, you can’t interact with gpt4 without openai’s api, so you don’t know what they’re doing in between your request and inference.

But LLMs are deterministic when all the parameters are the same between inferences