News for hackers. Stuff that matters.
Re:Binance built a 100PB log service with Quickwit by rthnbgrredf on Thursday July 11, 2024 @12:15PM (#40938161)
For this purpose you would likely not buy ordinary consumer disks but rather bullet proof enterprise HDDs. Otherwise a signifcant amount of the 5556 disks would not survive the first year, assuming the are under constant load.
Re:Binance built a 100PB log service with Quickwit by pas on Thursday July 11, 2024 @03:17PM (#40939681)
quickwit's big advantage is that you can target it at something that speaks S3 and it will be happy. so ideally you delegate the whole storage story by hiring someone who knows their way around Ceph (erasure coding, load distribution) and call a few DC/colo/hosting providers (initial setup and the regular HW replacements).