https://ollama.com/library/deepseek-r1

> Probably better for them to misunderstand and run 7b than run 671b. [...] if you don't like how things are done on Ollama, you can run your own object registry, like HF does.

I use Ollama for prototyping and then move what I can to a vLLM set up

I think the main point that turned me off was how they have their custom way of storing weights/metadata on disk, which makes it too complicated to share models between applications, I much prefer to be able to use the same weights across all applications I use, as some of them end up being like 50GB.

I ended up using llama.cpp directly (since I am a developer) for prototyping and recommending LM Studio for people who want to run local models but aren't developers.

But again, if you find Ollama useful, I don't think there is any reasons for dropping it immediately.

    ollama has their own way of releasing their models. 
    when you download r1 you get 7b. 
    this is due to not everyone is able to run 671b. 
    if its missleading then more likely due to user not reading.  

If you want the actual "lighter version" of the model the usual way, i.e. third-party quants, there's a bunch of "dynamic quants" of the bona fide (non-distilled) R1 here: https://unsloth.ai/blog/deepseekr1-dynamic. The smallest of them is just able to barely run on a beefy desktop, at less than 1 token per second.

DS has more or less been ignored for a very long time before this.

You think Ollama is purposefully using misleading naming because they're mad about DeepSeek? What benefit would there be for Ollama to be misleading in this way?

There is no benefit I think.

How do you get accurate information on the template structure?

Not that I don't believe you (I do, and I think I've seen them correct this before too), but you happen to have specific examples when this happened?

More recently deepseek 2 had a space after the assistant turn, causing issues with output quality and language https://www.reddit.com/r/LocalLLaMA/comments/1dko6rp/if_your...

Sadly, Ollama has a bit of a confusing messaging about this, and it isn't super obvious you're not actually testing the model that "comes close to GPT-4o" or whatever the tagline is, but instead testing basically completely different models. I think this can explain the mismatch in expectation vs reality here.

For such small models I would recommend specialized models only. Like Deep Seek Coder. But I think that one is lagging behind the state of art now.

Seems the explanation is the deepseek-r1 models I was using are not, in fact, deepseek-r1. Thanks all for the heads up.

Like everything. You do not even mention your machine spec, so I'm assume you just pick the ones that fit, which probably the quant versions.

Quant versions of the "small" models do not perform that well. Not the way you expected them to be.

They're also hyper sensitive to what I'd describe as geometric congruency between input and output: your input has to be able to be decompressed into final form with basically zero IQ spent on it, as if the input were zipped version of as yet existing output that the LLM simply macro-expanded.

R1 is just an improved LLM, nothing groundbreaking in those specific areas. Common limitations of LLMs still apply.

IMO, the layman's model of LLM should be more of predictive text than AI. It's a super fast keyboard that types faster than, not better than, your fingers.

I'm not sure where you get this generalization from. It seems like most local models you can run locally today on consumer hardware are kind of at that level, at least in my experience. But then you have things like o1 "pro mode" which pretty much allowed me to program things I couldn't before, and no other LLM until o1 could actually help me do.

Verilog is relatively niche as far as programming languages go, so I’m not surprised that you’d have trouble getting good output generally. You can only train the model on so much stuff, and there is probably limited high quality training data for verilog. It’s possible the model planners just decided not to prioritize this data in the training set. 8b sized models will especially struggle to have enough knowledge about niche topics to reason over it. Anything that small is really just a language tool for NLP tasks unless it’s trained specifically to do something.

All that said, your comment does illustrate a misunderstanding with the “thinking” models. They always output a long monologue on what to say, for anything, even “hello”. It’s a different skill to prompt and steer them in the right direction. Again, small models will be worse at everything, even being directed in the right direction.

TLDR: I think you need to find a new model, or at least try the “full” version through the web app or API first.

the mind model behind it very different then that of "normal" programming languages, so less reuse of learned knowledge from other places ("knowledge" for a lack of better wording)

Deepseek 8b is based on Meta's Llama3

I would say 70b doesn't worth the cost in comparison to 32b except coding... but if you want a model for coding, then you should try a model specifically trained for that, like Deepseek-Coder or Mistral's Codestral.

Deepseek-r1 is considered a general use AI. It would do good enough at many topics but won't excel on everything.