News for hackers. Stuff that matters.
Re:Why Not Comments by kristiandupont on Wednesday September 11, 2024 @06:09AM (#41509859)
You must have extremely busy logs, then? Do you also do this with all the code that runs in loops?
Re:Why Not Comments by XorNot on Wednesday September 11, 2024 @06:38AM (#41510036)
Do you routinely run production services at debug or trace log level?

The point is there's a big difference between "we've got a problem, we need to add logging and redeploy to try and isolate it" versus "we might have a problem, bump the logging level up on that service to see what's going on" (which with the right system you can do without even restarting).

Re:Why Not Comments by kristiandupont on Thursday September 12, 2024 @06:08AM (#41519218)
No, I don't. But even when I turn debug/trace on, I want the logs to have a reasonable signal to noise level.
Re:Why Not Comments by __MatrixMan__ on Wednesday September 11, 2024 @03:03PM (#41514334)
I'm not who you're asking, but I do the same

Re: busy, no because I only enable debug logging when I need it

Re: loops, I wrote a deduplicating logger, so if there are 200 identical messages from within a loop I see:

    {event: "thing happened"}

    {event: "thing happened",
     duplicates: 100, 
     since: "2024-09-11T01:05:00Z",
     reason: "count reached"}
That is, supposing that it's set to saturate at 100 events and they all happen fast enough that it doesn't reset the count. In this case it'll log another batch of 100 on the 201st event.

I think it's important to log the first one immediately, just in case you want to alert on that message. It wouldn't do to wait for the other 99 before raising the alarm.

I wrap my loggers in deduplicators only when I'm about to hand them off to a loop. Otherwise it's the normal ones.