News for hackers. Stuff that matters.
Re:We gave terabytes of CI logs to an LLM by iririririr on Friday February 27, 2026 @02:03PM (#47184225)
am i reading correctly that the compression is just a relational records? i.e. omit the pr title, just point to it?
Re:We gave terabytes of CI logs to an LLM by aluzzardi on Friday February 27, 2026 @02:33PM (#47184537)
There are 2 layers of compression:

- ZSTD (actual data compression)

- De-duplication (i.e. what you're saying)

Although AFAIK it's not "just point to it" but rather storing sorted data and being able to say "the next 2M rows have the same PR Title"