MakazhanAlpamys

homie lets keep building

yoo, it's good? how is it work?

good

i don't follow gpu prices.

the cheapest one is the one you already have.

more vram for the money is what i'd look for. usually used :)

you're right. it was the second, not the first. i already admitted that earlier in the thread :)

Those are format examples and test fixtures. Five to ten rows each. Not training data. You did spot a real problem though. Eight configs in `examples/configs` pointed at those fixtures as training data. Seven were still on the old schema and would not even parse. I've fixed that, added a README to the folder, and a test that parses every config so it doesn't quietly break again. Dataset size mostly depends on the task. Format or style, a few hundred examples is often enough. A task the model already half knows, usually a few thousand. New facts are often a bad fit for fine-tuning. I'd reach for RAG instead. I haven't measured how this changes with model size, so I'm not going to make up a number.

It isn't. Kazakh and Russian. I said this further down but that comment is dead so you would not have seen it. The later replies are mine, written by me.

Do not buy a 4 GB card for this. Mine is an RTX 3050 Laptop, I picked it because it is boring hardware that many people already have. If you are buying, buy VRAM. At 0.5B where I could measure both, resident training was 1.43x faster than streaming. And if you do stream, system RAM matters more, the base sits there and has to page-lock. 16 GB is about the floor for 8B.

Because streaming only removes the decoder stack. The embeddings and lm_head stay resident, that is 2.10 GB of the 3.32 GB peak on 8B. And the logits tensor scales with batch x seq x vocab, not with depth.

So it goes from "whole model must fit" to "embeddings plus one layer plus logits must fit". That is why 8B works and why I did not try 14B.

The table on the site is the normal resident path, streaming is opt-in and BETA. Should be clearer, my fault.

Depends what you change. Format or style, few hundred rows is often enough. A task the model already half knows, few thousand.

New facts is where people waste a week. The model comes back wrong in a new way. Use RAG for facts.

Those are mine. I used an LLM for my replies and that was a bad call, I said so further down.

Writing them myself now.