News for hackers. Stuff that matters.
Re:AI is becoming 'yes-men on servers' by XCabbage on Monday March 10, 2025 @05:12AM (#43318365)
Obvious thought that I haven't tested: can you literally achieve this by getting it to answer in Dutch, or training an AI on Dutch text? Plausibly* Dutch-language training data will reflect this cultural difference by virtue of being written primarily by Dutch people.

* (though not necessarily, since the Internet is its own country with its own culture, and much training data comes from the Internet)

Re:AI is becoming 'yes-men on servers' by zoover2020 on Monday March 10, 2025 @05:31AM (#43318489)
I've tried Dutch answers and it is more than happy to hallucinate and give me answers that are very "American". Doesn't help that our culture is very inspired by the US pop culture as well since the internet.

Haven't tried prompt engineering with the Dutch stereotype, though.

Re:AI is becoming 'yes-men on servers' by berkes on Monday March 10, 2025 @09:39AM (#43320487)
That hardly works. Though from my limited experiments, claude's models are better at this than OpenAIs. OpenAI will, quite often, come with suggestions that are literal translations of "anglicist" phrases.

Such as "Ik hoop dat deze email u gezond vindt" (I hope this email finds you well), which is so wrong that not even "simple" translation tools would suggest this.

Seeing that OpenAIs models can (could? This is from a large test we did months ago) not even use proper localized phrases but uses American ones, I highly doubt it can or will respond by refusing answers when it has none based on the training data.

Re:AI is becoming 'yes-men on servers' by berkes on Tuesday March 11, 2025 @04:15AM (#43330299)
Some of the examples are still wrong. Nuanced, but a Dutch native will still frown at it.

But more importantly is that you limited the context a lot. As in: the scope, the prompt, is very narrow.

In our case, we were generating emails. Lines like greetings are but one of 20+ details in that mail and not even the most important ones. The prompts ever larger, the multishot examples ever more tuned. And then, one in a few hundred will turn up with these "horrible" translations.

We've now moved to a chain of models, where we generate emails in American (the creative part) and then use another model to translate them to Dutch (the non-creative but culturally aware part). This works much better as we can pick models that are good at one thing or tuned to do this one thing better (either by the LLMAAS provider, or by parameters such as temperature).

Re:AI is becoming 'yes-men on servers' by eru on Wednesday March 12, 2025 @08:55AM (#43342711)
> Some of the examples are still wrong. Nuanced, but a Dutch native will still frown at it.

Thanks, that why I posted the links here: I couldn't judge by myself how good these creations were.

> But more importantly is that you limited the context a lot. As in: the scope, the prompt, is very narrow.

Yes, I just did a very crude experiment.

Re:AI is becoming 'yes-men on servers' by pishpash on Monday March 10, 2025 @07:56PM (#43327614)
Internal vectors aren't in any particular language.
Re:AI is becoming 'yes-men on servers' by daemonologist on Tuesday March 11, 2025 @03:32AM (#43330058)
No, but theoretically, blunt responses might be more common in Dutch-language training data. A well-fit model would be expected to replicate that. (Basically similar to straight up asking it to be more blunt, except it has probably trained a lot more on _Dutch_ than on _someone just told me to be blunt_ so the effect might be more natural and more subtle.)
Re:AI is becoming 'yes-men on servers' by daemonologist on Tuesday March 11, 2025 @03:28AM (#43330036)
I wonder (if this works at all) if the effect might be stronger if you also prompted in Dutch, preferably written by a fluent speaker rather than machine-translated.