News for hackers. Stuff that matters.
Re:Ask HN: Where can I find practical comparative data regar... by BMSR on Wednesday March 13, 2024 @04:56AM (#39689324)
I'm also learning. The models get more accurate when they have more parameters, say 7b (7 billion parameters) vs 8x7b (56 billion parameters). They also take more time and resources at higher parameters. TheBloke at Huggingface uploads quantized models, which means they can run on lower spec computers but with a possible hit on quality, he offers multiple configurations per model depending on what you prefer. Big models can be too heavy and slow, the sweet spot is probably something like 13b. You can try different gguf models with this program: https://github.com/madprops/meltdown
Re:Ask HN: Where can I find practical comparative data regar... by lkrubner on Wednesday March 13, 2024 @06:08AM (#39689718)
Thank you for this. It's a good start. But also, there is so much on Hugging Face, how does anyone evaluate it all? It's not possible to personally test everything, and develop good intuitions about what might be useful in particular situations.

As an analogy, 10 years ago we had a lot of debates on Hacker News about various languages and frameworks: PHP versus Python and Rails versus Django. But where do I go nowadays for similar discussions about all of the tooling that is springing up around the AI and LLM and NLP space?

Re:Ask HN: Where can I find practical comparative data regar... by BMSR on Wednesday March 13, 2024 @06:41AM (#39689929)
I found this thread useful: https://old.reddit.com/r/LocalLLaMA/comments/17fhp9k/huge_ll...

It compares different kinds of models at several tiers.

If you find a model interesting you can then look for it on HF.

Re:Ask HN: Where can I find practical comparative data regar... by lkrubner on Wednesday March 13, 2024 @03:17PM (#39696086)
Thank you again.