My locally-hosted llama3 actually craps itself if I ask it to answer in other languages. It's pretty hilarious. Has been working flawlessly (and impressively fast) for everything in English, then does hilarious glitches in other languages.

Eg right now to show it here, I say "Write me a poem about a digital pirate in Danish":

Digitalen Pirat

På nettet sejler han, En digital pirat, fri og farlig. Han har øjnene på de kodeagtige Og hans hjerne er fuld af ideer.

Hans skib er en virtuel børs, Hvor dataenes vætætø Tø Tø Tø Hø T Ø T Ø T Ø T Ø T Ø T 0 Ø T 0 Ø T 0

Edit: Formatting is lost here, but all those "T" and "Ø" etc are each on their own line, so it's a vomit of vertical characters that scrolls down my screen.

Trying the same on https://llama3.replicate.dev/ with Llama 3-70B gives a perfectly fine response with a long poem in Danish. And then it even translates it to English before concluding the response.

The training data is 95% English, foreign language is not going to be its strongest strength.