News for hackers. Stuff that matters.

A comment on Discord, or the Death of Lore. Parent comment

Re:Discord, or the Death of Lore by jhgg on Monday March 06, 2023 @11:21PM (#35051508)
Because it would be obscenely expensive (in terms of computational resources) to search history via grep. Hence why we use an inverted index.
[ Parent | on HN ]
Re:Discord, or the Death of Lore by rom-antics on Monday March 06, 2023 @11:35PM (#35051603)
This is megabytes (at most) of text we're talking about, not gigabytes. And ripgrep is absurdly fast. Grepping a 5MB text file should be pretty much instantaneous.
[ Parent | on HN ]
Re:Discord, or the Death of Lore by jhgg on Tuesday March 07, 2023 @12:55AM (#35052071)
Do you know of any service that is running at scale out there that stores their data as 5mb plain text files?
[ Parent | on HN ]
Re:Discord, or the Death of Lore by selfhoster11 on Tuesday March 07, 2023 @03:25AM (#35052969)
The “text files” don’t have to be the primary backing store - merely a derived cache file that can be periodically refreshed with new content. It doesn’t have to be text either - SQLite is pretty structured if you need more features in a minimal container too.
[ Parent | on HN ]