News for hackers. Stuff that matters.
Re:We gave terabytes of CI logs to an LLM by aluzzardi on Friday February 27, 2026 @02:33PM (#47184537)
There are 2 layers of compression:

- ZSTD (actual data compression)

- De-duplication (i.e. what you're saying)

Although AFAIK it's not "just point to it" but rather storing sorted data and being able to say "the next 2M rows have the same PR Title"

There are no comments on this comment yet.