Claude Code: connect to a local model when your quota runs out

Re:Claude Code: connect to a local model when your quota run... by polynomial on Thursday February 05, 2026 @01:23AM (#46896361)

Depending on what your usage requirements are, Mac Minis running UMA over RDMA is becoming a feasible option. At roughly 1/10 of the cost you're getting much much more than 1/10 the performance. (YMMV)

https://buildai.substack.com/i/181542049/the-mac-mini-moment

[ Parent | on HN ]

Re:Claude Code: connect to a local model when your quota run... by danw1979 on Thursday February 05, 2026 @07:26AM (#46898889)

I did not expect this to be a limiting factor in the mac mini RDMA setup ! -

> Thermal throttling: Thunderbolt 5 cables get hot under sustained 15GB/s load. After 10 minutes, bandwidth drops to 12GB/s. After 20 minutes, 10GB/s. Your 5.36 tokens/sec becomes 4.1 tokens/sec. Active cooling on cables helps but you’re fighting physics.

Thermal throttling of network cables is a new thing to me…

[ Parent | on HN ]

Re:Claude Code: connect to a local model when your quota run... by cat_plus_plus on Thursday February 05, 2026 @10:48AM (#46900970)

I admire patience of anyone who runs dense models on unified memory. Personally, I would rather feed an entire programming book or code directory to a sparse model and get an answer in 30 seconds and then use cloud in rare cases it's not enough.

[ Parent | on HN ]

Re:Claude Code: connect to a local model when your quota run... by polynomial on Thursday February 05, 2026 @12:26PM (#46902070)

Luckily we're having a record cold winter and your setup can double as a personal space heater.

[ Parent | on HN ]