News for hackers. Stuff that matters.
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by jesterson on Tuesday November 21, 2023 @02:03AM (#38360330)
Does anyone have any feedback on using those open-source models with any language except English? Particularly non-western group of languages like korean/japanese/chinese?

Will assess myself but wonder if anyone tried.

Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by clarionbell on Tuesday November 21, 2023 @03:54AM (#38361130)
They work, but they need way more tokens to express themselves. Still, it beats paying OpenAI for the privilege.
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by 3Sophons on Tuesday November 21, 2023 @02:15AM (#38360419)
For Chinese, there are this Yi Model by 0.1 AI Yi 34B https://www.secondstate.io/articles/yi-34b/ Uncencored Yi: https://www.secondstate.io/articles/dolphin-2.2-yi-34b/

and also Baichuan https://www.secondstate.io/articles/baichuan2-13b-chat/

There is an Arabic one called Jais mentioned by Satya a few days ago on the Microsoft dev day would like to try out

Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by wahnfrieden on Tuesday November 21, 2023 @06:22AM (#38362191)
Yi has weird license
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by 3Sophons on Tuesday November 21, 2023 @08:08AM (#38363149)
emmm do you mean not truly open source
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by wahnfrieden on Tuesday November 21, 2023 @12:21PM (#38366740)
You must send them an email to get commercial use permission. It does not look like a lawyer wrote their license either
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by 3Sophons on Tuesday November 21, 2023 @09:20PM (#38373811)
I think you have to do similar things with Baichuan too. Which is also a bit pseudo open source
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by nmfisher on Tuesday November 21, 2023 @02:34AM (#38360528)
Alibaba's Qwen is excellent for Chinese.
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by 3Sophons on Tuesday November 21, 2023 @09:22PM (#38373835)
For Qwen there is this the CausualLM 14B model, based on llama2 architecture but with Qwen 14B model weights.
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by nmfisher on Tuesday November 21, 2023 @10:25PM (#38374382)
Oh thanks, I didn't know about that. Will check it out!
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by 3Sophons on Wednesday November 22, 2023 @12:59AM (#38375218)
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by numpad0 on Tuesday November 21, 2023 @04:27AM (#38361394)
It's probably better to switch to suitable language-specific models. GPTs sound translated in non-native languages, and also has difficulty separating CJK due to that Unicode quirk.
Re:Self-Hosting Open Source LLMs: Cross Devices and Local De... by axegon_ on Tuesday November 21, 2023 @04:40AM (#38361486)
A bit of a hit and miss really. I haven't fiddled with languages all that much, but I was kinda curious to try a bunch of models with Spanish(the only large language other than English in my toolbox) about a month or two ago. Aguila 7B was the one that really stood out but then again it shouldn't come as a surprise - it is practically Falcon 7B trained on Spanish data.