News for hackers. Stuff that matters.
Re:I want everything local – Building my offline AI workspace by imoverclocked on Saturday August 09, 2025 @02:31AM (#44844501)
Everyone benefits from focusing on efficiency and finding better ways of doing things. Those people with 4TB+ of fast storage can now do more than they could before as can the "bottom tier."

It's a breath of fresh air anytime someone finds a way to do more with less rather than just wait for things to get faster and cheaper.

Re:I want everything local – Building my offline AI workspace by PeterStuer on Saturday August 09, 2025 @02:45AM (#44844552)
Of course. And I am not arguing against that at all. Just like if someone makes an inference runtime that is 4% faster, I'll take that win. But would it be the decisive factor in my choice? Only if that was my bottleneck, my true constraint.

All I tried to convey was that for most of the people in the presented scenario (personal emails etc.) , a 50 or even 500GB storage requirement is not going to be that primary constraint. So the suggestion was the marketing for this usecase might be better spotlighting also something else.

Re:I want everything local – Building my offline AI workspace by ricardobeat on Saturday August 09, 2025 @06:03AM (#44845298)
You are glossing over the fact that for RAG you need to search over those 500GB+ which will be painfully slow and CPU-intensive. The goal is fast retrieval to add data to the LLM context. Storage space is not the sole reason to minimize the DB size.
Re:I want everything local – Building my offline AI workspace by brookst on Saturday August 09, 2025 @09:45AM (#44846404)
You’re not searching over 500GB, you’re searching an index of the vectors. That’s the magic of embeddings and vector databases.

Same way you might have a 50TB relational database but “select id, name from people where country=‘uk’ and name like ‘benj%’ might only touch a few MB of storage at most.

Re:I want everything local – Building my offline AI workspace by ricardobeat on Saturday August 09, 2025 @06:54PM (#44851128)
That’s precisely the point I tried to clear up in the previous comment.

The LEANN author proposes to create a 9GB index for a 500GB archive, and the other poster argued that it is not helpful because “storage is cheap”.