News for hackers. Stuff that matters.
Re:Llama 3.1 in C by AMICABoard on Tuesday July 23, 2024 @11:53PM (#41053463)
Okay but hold your horses. Still a bit buggy.

Sample output:

Meta's Llama 3.1 models can output multilingual text which is awesome. Here are some examples output of 8 bit quantized 8b model with 100 token output (-n 100)... Quantization creates some brain damage.

English

My cat is funny. "Funny cat," I say, walking up to it. "What are you up to?" It sits up straight and looks at me with a tilted head, as if to say, "What's wrong with you?" Sometimes I just have to laugh at how funny a cat can be. So I say, "Okay, you're funny. I'll give you some treats." It stretches out a little and I give it some treats. It eats them up quickly and starts

German

Besitzen Sie einen Amiga 500? Wenn nicht, werden Sie wissen, dass dies ein Computer war, der im späten 1980er und frühen 1990er Jahren für Spiele verfügbar war, die für Personen mit bestimmten Körperverletzungen gedacht waren. Manchmal müssen wir uns an frühere Zeiten erinnern, die, wie wir jetzt wissen, schwierig waren. Hier ist ein Link, der meine Geschichte bespre

French

Le vin français est, à bien des égards, un vin des origines, car il a joué un rôle important dans l'histoire de la France". La réputation des vins de France repose principalement sur leurs qualités gustatives et la gestion des vignobles contrôlée, ce qui rend le vin français un "produit d'exception". La France est donc leader mondial de la production de vin, avec 25 % des exportations mon

Thai

แมวของฉันตลกชอบเล่นบนม้วนกระดาษ และฉันก็ไม่แน่ใจว่าควรจะยินยอมที่จะให้เล่นหรือไม่

เมื่อเวลาผ่านไป ฉันเห็นว่าแมวของฉันเล่นม้วนกระดาษเป็นระยะ ๆ ฉันจึงตัดสินใจที่จะลองปรับเปลี่ยนเกมให้สนุกขึ้น

Hindi

मेरी बिल्ली बहुत मज़ाया है और वह हमेशा अपनी शारीरिक गतिविधियों से मुझे मजाक करती है। वास्तव में, जब वह अपनी खिलौनों की चपपेट में आती है तो वह विशेष रूप से क्लासिक बन जाती है। इसके अलावा, वह एक छोटी सी च

Re:Llama 3.1 in C by FrostKiwi on Wednesday July 24, 2024 @02:32AM (#41054172)
> Quantization creates some brain damage.

Love the wording.

Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @04:07AM (#41054675)
Maybe we should make "Brain Damage Factor" a official term to denote how much types of quantizations degrade output compared to unquantized.:)
Re:Llama 3.1 in C by Y_Y on Wednesday July 24, 2024 @04:55AM (#41054977)
I think that's known as degradation, but I think the brain damage metric could be usefully applies to the poor sods who try to get the quantization to work in the first place.
Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @06:27AM (#41055493)
Lol :)
Re:Llama 3.1 in C by impossiblefork on Wednesday July 24, 2024 @04:55AM (#41054982)
There's actually an old paper titled Optimal Brain Damage, where they don't try to find optimal quantizations, but optimal sparse versions of a models-- i.e. where some weights are set to zero.
Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @06:28AM (#41055503)
Interesting, I heard something like that, but now I must read about it.
Re:Llama 3.1 in C by nickpsecurity on Wednesday July 24, 2024 @10:06AM (#41057264)
Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @10:27AM (#41057459)
Thanks, I'll read it up. Interesting.
Re:Llama 3.1 in C by nickpsecurity on Wednesday July 24, 2024 @10:35AM (#41057537)
That was really for them. You’re out there building neat stuff. Your talent might warrant looking into AdderNets and Bitnets which might get the cost down. There’s also some brain-inspired designs.

I don’t think many people have implemented such things. You might discover something new experimenting with them.

Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @12:35PM (#41058815)
:)
Re:Llama 3.1 in C by actionfromafar on Wednesday July 24, 2024 @03:30AM (#41054454)
Einen Amiga 500?

Shut up and take my money!

Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @04:05AM (#41054665)
Ja Amiga 500! My first computer. Still in love with her...:)
Re:Llama 3.1 in C by aninteger on Wednesday July 24, 2024 @08:44AM (#41056479)
> My cat is funny. "Funny cat," I say, walking up to it. "What are you up to?" It sits up straight and looks at me with a tilted head, as if to say, "What's wrong with you?" Sometimes I just have to laugh at how funny a cat can be. So I say, "Okay, you're funny. I'll give you some treats." It stretches out a little and I give it some treats. It eats them up quickly and starts

This is kind of like 3rd grade English. What would be required to go beyond that?

Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @10:26AM (#41057449)
Actually "My cat is funny" was the prompt it continued that. I got to fix some stuff to reflect meta's implementation and also fix the chat mode, then it would be usable. Will take a few days to do that.
Re:Llama 3.1 in C by trhway on Wednesday July 24, 2024 @09:35PM (#41063970)
>This is kind of like 3rd grade English

This is also how you'd communicate with a cat or dog. We [still] can't use complex long sentences in communications with them. And the LLM seems to be nicely reproducing that style and mindset of complete yet small sentence-bites you're in when communicating with your Good Boy or your Funny Cat.

Re:Llama 3.1 in C by AMICABoard on Thursday July 25, 2024 @02:48AM (#41065466)
Hmmm interesting take.
Re:Llama 3.1 in C by alex_smart on Thursday July 25, 2024 @02:15AM (#41065256)
The Hindi output is garbage.
Re:Llama 3.1 in C by AMICABoard on Thursday July 25, 2024 @02:47AM (#41065458)
Likely, cos other langs too. The fault may be my incomplete implementation.