To say anything as general as ‘the new file format is broken’ just means you either don’t understand the project basics or do not follow closely the commits.

Llamacpp underwent some major changes last few weeks. And following the commits it took few days to stabilise. Try now , works as bliss. And compared to other inference engines such as tinygrad - is much more versatile in options how to be run.

But you should be able to get it to do stuff just by telling it what you want. Note that it's completion tuned (not chat-tuned), so it should perform better on single-turn conversations.

  ollama run phind-codellama "write c code to reverse a linked list"

Then comes the real question, which is "let's write fizz buzz so it generates at above 55Gbytes/second".

Previously discussed here on HN: https://news.ycombinator.com/item?id=29031488

I tried with ollama, installed from Homebrew, on my M1 Max with 64GB RAM.

I downloaded the phind-codellama model using

  ollama pull phind-codellama

  ollama run phind-codellama "write a production grade implementation of Sieve of Eratosthenes in Rust"

    Error: Post "http://localhost:11434/api/generate": EOF

Even though it worked to run ollama with some other models.

Is the version in homebrew not able to run phind-codellama?

The download (both as a Mac app and standalone binary) is available here: https://github.com/jmorganca/ollama/releases/tag/v0.0.16. And I will work on getting that brew formula updated as well! Sorry to see you hit an error!

I'm on a MacBook Pro M1 Max 64GB.

    % ollama --version
    ollama version 0.0.16
    % ollama run phind-codellama "$(cat prompt.txt)"
    ⠧   Error: Post "http://localhost:11434/api/generate": EOF
    % wc prompt.txt
         335    1481   11457 prompt.txt

Do you mean triple double quotes?

I created an issue, if you have an example prompt to add that would be helpful! https://github.com/jmorganca/ollama/issues/422

[1] https://github.com/go-skynet/LocalAI

But I'd like to also be able to run these models on my Linux desktop with two GPU's (a 2080Ti and a 3080Ti) and a Threadripper. How difficult would it be to set some of these up on there?

I personally use llama.cpp as the driver since I run CPU-only but another may be better suited for GPU usage. But then it's as simple as downloading the model and placing it in the directory.

I'm admittedly running Sonoma beta, that's probably why

I will try it on my Linux machine later tonight (currently AFK with laptop)

> write c code to reverse a linked list

The number of times I had to do it in real production code amounts to zero.

The number of times I had to piece code from poorly documented external services, conflicting product requirements, and running on complex and weird environments has been ... well ... multiple times a day for the past 20+ years.

"instruct-like prompts" is what you give a very junior engineer out of college, and then you have to carefully review their code.

  Pretend to be a commenter on hackernews. Respond to the comment below:

  [parent comment inlined]

  what is your response?
  "Wow, that's great to hear! It sounds like you had a really positive experience with the 34B last night. I'm also excited to see what's in store for Phind and its potential applications. Have you tried using the 34B for any specific tasks or projects yet? And do you think the text completion feature would be useful for your use case if it were improved further?"

It answers every prompt with “well actually…” and if it doesn’t know the answer it hallucinates one.

And since that question has been answered in the negative, I'm inclined to agree.