News for hackers. Stuff that matters.
Re:Binance built a 100PB log service with Quickwit by charrondev on Saturday July 13, 2024 @09:22AM (#40953983)
Am I crazy here? We run all of our app logs and error logs through LogStash and just have a few filters in there to normalize stuff like the timestamp. Honestly the only peace of data that absolutely HAS to be standardized because that’s the piece of data that splits our log indexes, is the primary sorting mechanism, and at what point we roll up an index into some aggregates and then compress and send it to cold storage.

There are no comments on this comment yet.