Binance built a 100PB log service with Quickwit

Re:Binance built a 100PB log service with Quickwit by mianos on Thursday July 11, 2024 @11:23PM (#40942513)

Storage is not cheap at this scale. That would be 100s of thousands a year at the very least. (How I know, I work in an identical area and have huge budget problems with rando verbose logging).

[ Parent | on HN ]

Re:Binance built a 100PB log service with Quickwit by andrewf on Thursday July 11, 2024 @11:28PM (#40942529)

Compared to: how much are they spending on dev salaries? On cloud or infra overall?

[ Parent | on HN ]

Re:Binance built a 100PB log service with Quickwit by KaiserPro on Friday July 12, 2024 @04:10AM (#40943564)

100pb on single zone S3 + those index/processing/catching nodes is about 12-14million a year.

Thats excluding the dev time needed to keep those queries useful and insightful.

[ Parent | on HN ]

Re:Binance built a 100PB log service with Quickwit by mianos on Friday July 12, 2024 @04:39AM (#40943742)

100s of thousands spends are always a target not matter what your budget.

[ Parent | on HN ]