Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on The Coming of Local LLMs. Parent comment
> What I'm wondering is how they fed the documents, as all those LLMs have limitations on the input sizes.
It's like file hashing at scale, you don't have to read the whole stream for every file, just the first 1024/2048 bytes (or first few paragraphs).
(This works for classification and sorting, less so for summarization.)
There are no comments on this comment yet.
> What I'm wondering is how they fed the documents, as all those LLMs have limitations on the input sizes.
It's like file hashing at scale, you don't have to read the whole stream for every file, just the first 1024/2048 bytes (or first few paragraphs).
(This works for classification and sorting, less so for summarization.)