Re:Binance built a 100PB log service with Quickwit
by londons_explore on Thursday July 11, 2024 @12:41PM
(#40938386)
When you have metrics, you should also keep sampled logs.
Ie. 1 per million log entries is kept. Write some rules to try and keep more of the more interesting ones.
One way to do this is to have your logging macro include the source file and line number the logline came from, and then, for each file and line number emit/store no more than 1 logline per minute.
That way you get detailed records of rare events, while filtering most of the noise.
Re:Binance built a 100PB log service with Quickwit
by hooverd on Thursday July 11, 2024 @01:50PM
(#40938993)
There are also different types of logs. Maybe you want every transaction action but don't need a full fidelity copy of every load balancer ping from the last ten years.
Ie. 1 per million log entries is kept. Write some rules to try and keep more of the more interesting ones.
One way to do this is to have your logging macro include the source file and line number the logline came from, and then, for each file and line number emit/store no more than 1 logline per minute.
That way you get detailed records of rare events, while filtering most of the noise.