elsombrero

The risky part is in the agent/harness and what tools it has access to.

You don't need to give GPU passthrough to the VM running the agent/harness.

There is still a risk of a prompt messing with the inference server, but I think that's a much lower risk compared to an agent doing whatever on its own.

generally, the less parameters, the less knowledge they have.

https://github.com/homecluster-dev/homelab-autoscaler

https://autoscaler.homecluster.dev

Works with any mechanism to turn on and off nodes(IPMI, WoL...) I have some nodes that I turn on and off via a curl to homeassistant to the power plug.

it's a transparent proxy that automatically launches your selected model with your preferred inference server so that you don't need to manually start/stop the server when you want to switch model

so, let's say I have configured roo code to use qwen3 30ba3b as the orchestrator and glm4.5 air as coder, roo code would call the proxy server with model "qwen3" when using orchestrator mode and then kill llama.cpp with qwen3 and restart it with "glm4.5air"

I read a lot of good comments on r/localllama, with most people suggesting qwen3 coder 30ba3b, but I never got it to work as well as GLM 4.5 air Q1.

As for using Q2, it will fit in vram, but with very small context or spill over to RAM, but with quite an impact on speed depending on your setup. I have slow ddr4 ram and going for Q1 has been a good compromise for me, but YMMV.

I also tried to use it with claude code with claude code router and it's pretty fast. Roo code uses bigger contexts, so it's quite slower than claude code in general, but I like the workflow better.

this is my snippet for llama-swap

``` models: "glm45-air": healthCheckTimeout: 300 cmd: | llama.cpp/build/bin/llama-server -hf unsloth/GLM-4.5-Air-GGUF:IQ1_M --split-mode layer --tensor-split 0.48,0.52 --flash-attn on -c 82000 --ubatch-size 512 --cache-type-k q4_1 --cache-type-v q4_1 -ngl 99 --threads -1 --port ${PORT} --host 0.0.0.0 --no-mmap -hfd mradermacher/GLM-4.5-DRAFT-0.6B-v3.0-i1-GGUF:Q6_K -ngld 99 --kv-unified ```

With full charge on average I get around a day or two without recharging with mostly firefox, youtube, messaging apps open and some light gaming.

Battery saving mode gets me almost 1 hour more of usage when I'm at 15%

Haven't really had any signal problems and 5g isn't that much of an improvement around where I live so it's not really a concern for me but ymmv

And then there's this

> Here’s what you do:

>

> Read this book.

> Do what it says.

Bad self-help marketing fluff piece without any substance to it...

https://arxiv.org/abs/1808.10250