shelajev
A real use case for local models?
Building Custom Docker Sandboxes
Agent Builder and Runtime by Docker Engineering
from the article: > [Note 8: Informally, if A strongly happens before B, then A appears to be evaluated before B in all contexts. — end note]
this is the Java happens-before, right? What's the non-strong happens-berfore in C++ then?
it builds as a docker image which has stockfish and maia (maiachess.com) together with different weights so it can simulate lower-level players.
It was a fun exercise, I tried a bunch of local models with this MCP server, which isn't particularly optimized, but also doesn't seem that bad. And the results were quite disappointing, they often would invent chess related reasoning and mess up answering questions, even if you'd expect them to rely on the tools and have true evaluation available.
It was also fun to say things: fetch a random game by username 'X' from lichess, analyze it and find positions which are good puzzles for a player rated N.
and see it figure out the algorithm of tool calls: - fetch the game - feed the moves to stockfish - find moves where evaluation changed sharply - feed it to maia at strength around N and to stockfish - if these disagree, it's probably a good puzzle.
I don't think I got to have a working setup like that even with managed cloud models. Various small issues, like timeouts on the MCP calls, general unreliability, etc. Then lost interest and abandoned the idea.
I should try again after seeing this thread
really. Learning should [never] be fun - https://news.ycombinator.com/item?id=42099596
Building AI Agents with Goose and Docker
MCP Horror Stories: The Security Issues Threatening AI Infrastructure
Voxtral WebGPU
AI workflows, vibe coding and such only emphasize the need for proper test suites and for a measured release process, where you don't actually delete your production databases on the whim.
Only if people (end-users) actually care. If you're trying to build against the momentum but misjudge your values for the community values, then you're in for a disappointment.
The model itself is fun to think about: preventing failures by stuffing more cheese into the system. If you're interested, the classic example of the cheese failure is Chernobyl, where many different things had to fail in order to become a catastrophe.
and while running the code might faster, it's unclear whether that approach scales well. Sending an MCP tool command to click the button that says "X", is something a small local LLM can do. Writing complex code after parsing significant amount of HTML (for correct selectors for example) probably needs a managed model.
Models in biology: accurate descriptions of our pathetic thinking
Tool Calling with Local LLMs: A Practical Evaluation
Why Docker Chose OCI Artifacts for AI Model Packaging
Behind the scenes: How we designed Docker Model Runner and what's next
This way you're in control of what it means the code works, and formalize the acceptance criteria. Instead of feeding it a potentially incorrect implementation, and asking it to come up with a set of requirements what it should actually do.