you'll also notice these articles rarely specify their context window in tokens, because it is small, usually 30k to 70k tokens and it gets slower as it fills up.
Was because I am back to using Linux as my workstation.
My Mac Mini is now a headless server for llama.cpp.
So, you are right that for these workloads , I would not be using the Mac Mini for k8s AND llama.
Another thing going against using a Mac for Linux containers is that there are no solutions that I know that properly manages memory : memory is given to the Linux vm , but never fluctuates if the needs in the vm are less than the initial request.
I know Orb Stack does that but is it proprietary. I think UTM does it , but not sure I would use UTM instead of Lima, Colima , multipass , etc to run containers.
48G RAM is pretty useful if you want to run k8s locally for tests / exploration