News for hackers. Stuff that matters.
Re:Training great LLMs from ground zero by dauertewigkeit on Wednesday March 06, 2024 @03:42PM (#39621148)
I don't think most of them have any kind of secret sauce. I think the founders hope to get bought out simply for being able to train "near-SOTA" LLMs. I guess achieving that level of skill and infra could be valuable enough to build upon.
Re:Training great LLMs from ground zero by joe_the_user on Wednesday March 06, 2024 @04:45PM (#39622021)
Sure, that's also a factor but I'd say it reinforces my main point.
Re:Training great LLMs from ground zero by DeepChill on Wednesday March 06, 2024 @06:50PM (#39623352)
Good point, so the only real differentiator would be the size & quality of the data being fed and the fine tuning done on the model? I wonder what else differentiates LLMs from each other
Re:Training great LLMs from ground zero by llm_trw on Wednesday March 06, 2024 @07:46PM (#39623744)
Also getting a golden ticket.

Golliath 120b is still the best open source model and no one knows why since it's just two llama2 60b glued together.

Re:Training great LLMs from ground zero by Iulioh on Wednesday March 06, 2024 @07:22PM (#39623571)
Alignment and censorship ?
Re:Training great LLMs from ground zero by pests on Wednesday March 06, 2024 @07:54PM (#39623791)
Alignment just means making it do what you want. LLMs just continue the sequence, the chat questions and response style we have now is an example of alignment (to what humans want).
Re:Training great LLMs from ground zero by eru on Thursday March 07, 2024 @12:15AM (#39625302)
Alignment can mean making sure your LLM doesn't continue the sequence in embarrassing ways, eg by spouting politically incorrect sequences of words (even though those might have been common in the training data).
Re:Training great LLMs from ground zero by friendzis on Thursday March 07, 2024 @02:24AM (#39626138)
In what way does this do more good than harm?
Re:Training great LLMs from ground zero by eru on Thursday March 07, 2024 @08:40AM (#39628899)
In the sense of people caring about their models not saying embarrassing things?

Different people have different goals, and they don't necessarily align with yours.

Re:Training great LLMs from ground zero by friendzis on Friday March 08, 2024 @01:42AM (#39638448)
Since the entity releasing the model obviously has certain goals aligning/censoring model in some ways is good for their particular short-term goal.

In the grand scheme these alignments are harmful as they place a reality distortion field. Authors create model of what language is and then contort that model to fit an opinionated idea of what language should be. Smells a bit Orwellian, right?

Re:Training great LLMs from ground zero by eru on Friday March 08, 2024 @06:28AM (#39640073)
> Smells a bit Orwellian, right?

No, seems perfectly fine by me. You are already shaping your results by your selection of training data. Eg do you want to train a model that speaks English, or German, or both? Do you want to run your training data past a spam filter first? Do you want to do a character based model, or one of those weird encodings that is popular with LLMs these days?

Doing some other procedures afterwards to make sure your LLM doesn't say embarrassing things is small fries by comparison.

Also it's good practice for trying to get alignment with more important values (like "don't kill all humans") later when models might get powerful enough to be able to kill all humans.

Playing some little games where OpenAI tries to keep you from making their model say embarrassing things, and people keep trying to make it say embarrassing things, is a good low stakes practice ground.

Re:Training great LLMs from ground zero by pests on Friday March 08, 2024 @08:45PM (#39648678)
I agree but this entire conversation misses my point that "alignment" originally only meant making the LLM act as you want it.

A GPT that hasn't been aligned does not work how we expect - you give it a prompt, and it will autogenerate until it reaches an end state.

To even make the GPT answer the question in the prompt, and not autocomplete it into nonsense, is an example of alignment.

It took a lot of fine tuning and data curation to get ChatGPT up to its current chat-like interface.

But this is not the only alignment you can do. The original Transformer paper was about machine translation, turning the prompt into the translated text. Once it was done it was done.

We could choose to have the model do something else, say translate the prompt into 5 languages at once instead of one, just as an example. This would be another alignment decision.

There is nothing political or selection bais or anything inherent to the original definition, its only recently "alignment" has morphed into this "align with human morals" concept.

Even in the Andrej Karpathy's build-your-own-gpt YT video, which is highly talked about around here, he uses the phase like this. The end of the video you are left with a GPT, but not a question-and-response model, and he says it would need to be aligned to answer questions like ChatGPT.

Re:Training great LLMs from ground zero by imtringued on Friday March 08, 2024 @10:36AM (#39642020)
There was a guy who followed a tutorial about how to fine tune mistral with DPO, who has zero computer science skills and his model ended up at the top of the hugging face leader board among the opensource models with 7 billion parameters. Some random guy managed to outdo the creators of the LLM.