News for hackers. Stuff that matters.
Re:Show HN: Open-Source DocumentAI with Ollama by simonw on Saturday March 08, 2025 @10:44AM (#43301051)
Sadly, the hardest part of running local models with tools like Ollama appears to be longer context prompts.

Models that respond really quickly to a short sentence prompt need vastly more RAM and CPU/GPU time for significantly longer inputs. I'm finding this really damages their utility for me.

There are no comments on this comment yet.