News for hackers. Stuff that matters.
Re:Things we learned about LLMs in 2024 by layer8 on Tuesday December 31, 2024 @07:58PM (#42563089)
Thanks. Google AI Studio isn’t local, I think, is it? I’ll have to test this, but our project sizes and specification documents are likely to run into size limitations for local models (or for the clipboard at the very least ;)). And what I’d be most interested in are big-picture questions and global analyses.
Re:Things we learned about LLMs in 2024 by simonw on Tuesday December 31, 2024 @07:59PM (#42563096)
No, it's not. I've not seen any local models that can handle 1m+ tokens.

I haven't actually done many experiments with long context local models - I tend to hit the hosted API models for that kind of thing.