News for hackers. Stuff that matters.
Re:How F5Bot Slurps All of Reddit by codeplea on Monday July 30, 2018 @07:07PM (#17649015)
Thanks for the explanation. The APIs make a lot more sense in light of it. It seems odd they didn't just use Postgres indexing though. Do you know if they benchmarked it at any point?
Re:How F5Bot Slurps All of Reddit by anarazel on Monday July 30, 2018 @07:45PM (#17649287)
For a lot of queries EAV type schemas are really hard to index efficiently. E.g. searching for something like a = ? and b = ? where a and b are dynamic attributes you can't just have a multi-column index when using EAV. So you instead end up with two intermediate query results that then are intersected. If you need even semi-efficient querying EAV usually isn't the answer.
Re:How F5Bot Slurps All of Reddit by friendzis on Tuesday July 31, 2018 @06:26AM (#17651956)
But it is possible to use hybrid data store. Extract important attributes you want to search on efficiently and store them in properly structured and indexed relational database. Other attributes can still be stored EAV style in different schema/database.
Re:How F5Bot Slurps All of Reddit by Deimorz on Monday July 30, 2018 @07:14PM (#17649068)
I have no idea, this system was created years before I started working there.