News for hackers. Stuff that matters.
Re:Scaling our observability platform by embracing wide even... by ofrzeta on Saturday June 21, 2025 @07:22AM (#44336630)
> Dropping valuable observability data to save space is usually shortsighted

That's a bit of a blanket statement, too :) I've seen many systems where a lot of stuff is logged without much thought. "Connection to database successful" - does this need to be logged on every connection request? Log level info, warning, debug? Codebases are full of this.

Re:Scaling our observability platform by embracing wide even... by citrin_ru on Saturday June 21, 2025 @07:56AM (#44336844)
Probably not very useful for prod (non debug) logging, but it’s useful when such events are tracked in metrics (success/failure, connect/response times). And modern databases (including ClickHouse) can compress metrics efficiently so not much space will be spent on a few metrics.
Re:Scaling our observability platform by embracing wide even... by throwaway0665 on Saturday June 21, 2025 @07:40AM (#44336740)
There's always another log that could have been key to getting to the bottom of an incident. It's impossible to know completely what will be useful in advance.
Re:Scaling our observability platform by embracing wide even... by nijave on Saturday June 21, 2025 @02:10PM (#44339497)
Yes, it allows you to bisect a program to see the block of code between log statements where the program malfunctioned. More log statements slice the code into smaller blocks meaning less places to look.
Re:Scaling our observability platform by embracing wide even... by vidro3 on Saturday June 21, 2025 @07:32PM (#44341513)
in our app each user polls for a resource availability every 5 mins. do we really need "connection successful" 500x per minute? i dont see this as breaking up the logs into smaller sections. i see it as noise. i'd much rather have a ton of "connection failed" whenever that occurs than the "success" constantly