Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R...
by monegator on Monday August 03, 2026 @03:08AM
(#49152225)
Yes, that is the idea. did read the paragraph about the cli server but of course that goes beyon what i know about these tools, that's why i asked. Having a streaming chat via command line would be awesome, because then multiple users could use the machine at the same time (i think 2, max 3 on a 8GB device. It would really make our old mini useful again, instead of sitting in the corner taking space)
I think I might understand your use case. You ssh the Mac and want something like `ollama run` with an interactive chat in terminal. Am I right?
There is already experimental OpenAI-compatible server in this repo:
``` swift build -c release --product TurboFieldfareServer .build/release/TurboFieldfareServer \ --model scratch/gemma4.gturbo ```
After that a small terminal client can run inside the same ssh session and talk to `/v1/chat/completions`
The client needs to keep a messages array, add each user message, send the full array with `stream:true`, print SSE chunks until `[DONE]`, then add the response back to the array. `/reset` can clear it
There is a python example in the server docs. (https://github.com/drumih/turbo-fieldfare/blob/main/docs/OPE...)
It is non-streaming, but can be used as a starting point.
The server is still experimental and I am fixing some problems currently. But you can try to vibecode a simple terminal client around it.
If not, create an issue on Github and describe desired behaviour