aldielshala
aldielshala on Hacker News
Haven't tested on a Pi yet, llm.sql is still in alpha, focused on validating that SQLite can actually work for LLM inference and profiling memory usage. That said, 210MB peak RSS should fit comfortably on a Pi. In theory, any device that runs SQLite (which is almost every device) could run llm.sql. Planning to benchmark across different hardware as the project matures.
Posted by
aldielshala on Friday April 24, 2026 @07:28AM
from the show-hn dept.
Hi HN,I built llm.sql, an LLM inference framework that reimagines the LLM execution pipeline as a series of structured SQL queries atop SQLite.The motivation: Edge LLMs are getting better, but hardware remains a bottleneck, especially RAM (size and bandwidth).When available memory is less than the model size and KV cache, the OS incurs page faults and swaps pages using LRU-like strategies,...
Nice project. I'm also working on something that pushes SQLite well beyond its typical use case. It's encouraging to see more people exploring what SQLite can really do.
Curious how it handles 10K+ notes performance-wise, does it index everything or lazy-load?
Trying to use human attention, instead of Transformer attention.
Intent debt is a useful framing. A few comments explaining "why" instead of "what" would have saved hours of guessing.
Finally an AI that takes someone's job and nobody's upset about it.
My contribution today: fewer LLM calls, fewer GPU hours, less CO2.
Everyone's focused on Meta employees, but the real concern is normalization. If Meta does this and gets away with it, some companies may quietly roll out the same thing.
Yes, maybe context engineering (prompting is just one part of it) and soft skills.
Honestly, I doubt this data is as useful as they think.
Half my workday is me browsing random tabs while an AI agent does the actual work. They're going to train a model on alt-tabbing and scrolling HN/Twitter/Reddit.
$60B for a VSCode fork with AI integration...
It may show the value of the gap between vanilla LLM output and production-ready applications.
Communication, with both human and AI.