News for hackers. Stuff that matters.
Re:Structured Logging with Slog by mrweasel on Tuesday August 22, 2023 @02:04PM (#37226462)
Admittedly I'm not a huge fan of having:

  slog.Info("hello, world", "user", os.Getenv("USER"))
It's a little magical that "user" is a key. So what if you have multiple key-value pairs? Arguably it most likely going to be obvious which is the keys, but having every other value be a key and the rest values seems a little clumsy.

I really like Pythons approach where you can have user="value" it makes things a bit more clear.

Re:Structured Logging with Slog by grose on Tuesday August 22, 2023 @02:40PM (#37226893)
It’s not without precedence, for example: https://pkg.go.dev/strings#NewReplacer

I don’t mind it. You can use LogAttrs if you want to be explicit.

Although I do wonder if there’s anything tricky with the type system that is preventing something like this from being supported: https://go.dev/play/p/_YV7sYdnZ5V

Re:Structured Logging with Slog by gdprrrr on Tuesday August 22, 2023 @03:03PM (#37227143)
Is ordering of the keys guaranteed to be the same as in the literal?
Re:Structured Logging with Slog by grose on Tuesday August 22, 2023 @03:15PM (#37227302)
That’s a good point. I think it would be random without some sorting shenanigans.
Re:Structured Logging with Slog by kiitos on Wednesday August 23, 2023 @07:44AM (#37234394)
Order should be preserved up to the Handler. Some Handlers e.g. JSONHandler produce output with key order explicitly undefined.
Re:Structured Logging with Slog by ben0x539 on Tuesday August 22, 2023 @04:16PM (#37227981)
I think go loggers have tried to move away from passing log entries as maps for performance reasons.
Re:Structured Logging with Slog by mlhpdx on Tuesday August 22, 2023 @07:32PM (#37229950)
That seems like a problem that should be solved. Logging structured data is a very basic expectation.
Re:Structured Logging with Slog by infogulch on Tuesday August 22, 2023 @07:41PM (#37230017)
This is structured logging. Stubbornly insisting that "structured logging" === "map" is dumb and ignores a large fraction of use cases where performance matters.
Re:Structured Logging with Slog by mlhpdx on Tuesday August 22, 2023 @10:45PM (#37231297)
Are there languages that solve the “performance“ problem with maps? In fact, isn’t Go one of them?
Re:Structured Logging with Slog by infogulch on Tuesday August 22, 2023 @11:51PM (#37231637)
In this use case using maps doesn't solve any problem, requires at least one allocation, and requires hashing each key. This is not even an interesting discussion.
Re:Structured Logging with Slog by mlhpdx on Wednesday August 23, 2023 @12:23AM (#37231858)
I can see the case for flat logging, aka. key value logging, as an optimization for very, very performance critical code that needs to emit string logs. That however, isn’t mainstream in my experience. The far, far more common case is logging in code with complex data-driven behaviors where the data is structured (more than one level, not flat) and where forensic debugging via logs is a critical activity. If that’s not your world, you should only be interested in it if you’re interested in the community at large. If you’re not, that’s cool.
Re:Structured Logging with Slog by 59nadir on Wednesday August 23, 2023 @02:30AM (#37232482)
I think you seem to be arguing that the end result should be a "map"-like structure, whereas the other commenter is arguing about the interface to the logging library not being based on maps. These are not the same and taking maps in the interface is likely to incur allocations, yes. Having to specify your key-value pairs without maps is the only downside to not taking fully constructed maps in the interface.
Re:Structured Logging with Slog by mlhpdx on Wednesday August 23, 2023 @12:20PM (#37238360)
Adding the application/component code to do ”good logging” is tedious. Friction in the interface decreases the probability it will be done well, and consistently. I think the interface matters, and while it’s a second order problem it does impact system quality in the long run.

Just my opinions here. I don’t question the value of slog as it sits, just could have been better for the community at large is all.

Re:Structured Logging with Slog by yencabulator on Tuesday August 22, 2023 @07:50PM (#37230102)
Avoiding the map allocation & construction cost is way harder than avoiding the use of a map, like zap and slog.LogAttrs do.
Re:Structured Logging with Slog by masklinn on Wednesday August 23, 2023 @02:19AM (#37232431)
You don’t need LogAttrs to pass in Attr entries, it should work fine with normal functions

The reason it doesn’t use maps is that maps are significantly slower, TFA has an entire section on performances.

However if you prefer that interface and don’t mind the performance hit, nothing precludes writing your own Logger (it’s just a façade over the Handler interface) and taking maps.

Re:Structured Logging with Slog by eska on Tuesday August 22, 2023 @02:41PM (#37226895)
If you had read more than the introduction you’d have found a paragraph about this in the middle and multiple at the end.
Re:Structured Logging with Slog by masklinn on Tuesday August 22, 2023 @02:28PM (#37226737)
> It's a little magical that "user" is a key. So what if you have multiple key-value pairs?

You... add them afterwards? It's really just a plist (https://www.gnu.org/software/emacs/manual/html_node/elisp/Pr...), that's hardly novel. The method takes any number of parameters and pairs them up as key, value.

Or if you really hate yourself, you use LogAttrs with explicitly constructed Attr objects.

> I really like Pythons approach where you can have user="value" it makes things a bit more clear.

The trouble is, Go doesn't have keyword parameters so that doesn't work.

Re:Structured Logging with Slog by johnmaguire on Tuesday August 22, 2023 @05:23PM (#37228705)
It sure has maps though... logrus famously uses `logrus.Fields{"key": "value"}`
Re:Structured Logging with Slog by morelisp on Tuesday August 22, 2023 @05:38PM (#37228860)
And logrus is one of the slowest loggers by far, in part because of its heavy map usage.
Re:Structured Logging with Slog by dgsb on Wednesday August 23, 2023 @02:46AM (#37232587)
It's also the reflect on the value part which slows down things. It could be improved but it's hard to do that without breaking interface. It's so widely used that a major bump wouldn't that useful.
Re:Structured Logging with Slog by JonChesterfield on Wednesday August 23, 2023 @06:38AM (#37233985)
I'd expect it to be mostly called on map literals, at least a compile time constant set of keys. That should be amenable to a targeted compiler optimisation.
Re:Structured Logging with Slog by taeric on Tuesday August 22, 2023 @02:32PM (#37226782)
Plist or named arguments doesn't really make too big of a difference, to my eyes. I do recommend Mapped Diagnostic Context style approaches, if you can do it. Passing all arguments that you think may be useful to logs gets unwieldy quickly. Looks particularly absurd when folks start adding parameters to functions for the sole purpose of letting you log them.
Re:Structured Logging with Slog by nerdponx on Tuesday August 22, 2023 @02:37PM (#37226850)
[delayed]
Re:Structured Logging with Slog by packetlost on Tuesday August 22, 2023 @02:27PM (#37226727)
Yeah, I agree. Passing in an optional `map[string]string` or something would be better, but then you get into having to either pass in `nil` every time you don't have the extra data or needing an entirely different function for with vs without the map
Re:Structured Logging with Slog by masklinn on Tuesday August 22, 2023 @02:29PM (#37226750)
> Passing in an optional `map[string]string` or something would be better

It would definitely not be better from the point of view of

> We wanted slog to be fast.

Re:Structured Logging with Slog by packetlost on Tuesday August 22, 2023 @02:33PM (#37226807)
A better interface/API is really what I meant. The performance characteristics are probably worth the tradeoff.
Re:Structured Logging with Slog by returningfory2 on Tuesday August 22, 2023 @02:36PM (#37226849)
Passing in a map would require an extra allocation for the map memory for each log line. I think the performance would probably not be great?
Re:Structured Logging with Slog by packetlost on Tuesday August 22, 2023 @02:41PM (#37226900)
it depends. I believe map literals are stack allocated if they aren't shared across goroutines or globals.
Re:Structured Logging with Slog by masklinn on Wednesday August 23, 2023 @02:48AM (#37232606)
While the hmap struct can be stack allocated if it does not escape, I’m pretty sure the buckets still need to be heap-allocated. I do not believe hmap has a “flat structure” variant which could live entirely on the stack, though I could be wrong.
Re:Structured Logging with Slog by Ferret7446 on Wednesday August 23, 2023 @03:54AM (#37233033)
It's most definitely not. Logging is crucial for monitoring services, and making logging statements many times slower will either sink your service or push developers to avoid logging and making the service impossible to debug.

Most log metadata will be attached by libraries and middleware, so service/application devs won't even see most of it.

Re:Structured Logging with Slog by packetlost on Wednesday August 23, 2023 @11:42AM (#37237781)
I think you interpreted my comment the wrong way, I think the performance tradeoffs are worth the mediocre API design. My bad for making it ambiguous
Re:Structured Logging with Slog by baq on Tuesday August 22, 2023 @03:07PM (#37227201)
this is not uncommon. this is what I've been dealing with earlier today in Postgres:

  json_build_object ( VARIADIC "any" ) → json

  jsonb_build_object ( VARIADIC "any" ) → jsonb

  Builds a JSON object out of a variadic argument list.
  By convention, the argument list consists of alternating keys and values.
  Key arguments are coerced to text; value arguments are converted as per to_json or to_jsonb.

      json_build_object('foo', 1, 2, row(3,'bar')) → {"foo" : 1, "2" : {"f1":3,"f2":"bar"}}
Re:Structured Logging with Slog by paulddraper on Tuesday August 22, 2023 @07:15PM (#37229835)
I'm really used to writing them on new lines.

  json_build_object(
    'foo', 1,
    2, row(3, 'bar')
  )
Re:Structured Logging with Slog by jupp0r on Tuesday August 22, 2023 @03:41PM (#37227593)
I think Uber chose a better approach for their Go logging library called Zap [1]

    logger.Info("failed to fetch URL",
      // Structured context as strongly typed Field values.
      zap.String("url", url),
      zap.Int("attempt", 3),
      zap.Duration("backoff", time.Second),
    )
They also have zap.Error(err) which generates the "error" key as a convention.

[1] https://github.com/uber-go/zap

Re:Structured Logging with Slog by bpicolo on Tuesday August 22, 2023 @04:00PM (#37227804)
As mentioned in the article, Zap's SugaredLogger has roughly the same thing https://pkg.go.dev/go.uber.org/zap

    sugar.Infow("failed to fetch URL",
      "url", "http://example.com",
      "attempt", 3,
      "backoff", time.Second,
    )
Really slog presents mostly the same two APIs as Zap, with a single entrypoint.
Re:Structured Logging with Slog by Groxx on Tuesday August 22, 2023 @07:55PM (#37230136)
`[level]w` supports both key/value pairs and individual `zap.Type(key, value)` values combined fwiw, so for that part the API is essentially the same too
Re:Structured Logging with Slog by masklinn on Tuesday August 22, 2023 @04:24PM (#37228057)
Slog allows doing that by creating Attr structs directly.
Re:Structured Logging with Slog by sa46 on Tuesday August 22, 2023 @07:48PM (#37230083)
You can reproduce that example with slog verbatim by changing zap to slog.

For example: https://pkg.go.dev/golang.org/x/exp/slog#example-Group

Re:Structured Logging with Slog by hamandcheese on Tuesday August 22, 2023 @04:20PM (#37228015)
The same pattern can be seen in Java using Map.of(...)
Re:Structured Logging with Slog by candiddevmike on Tuesday August 22, 2023 @05:09PM (#37228561)
Today I'm pulling this out of a context object, can I still do that?
Re:Structured Logging with Slog by richardwhiuk on Tuesday August 22, 2023 @05:22PM (#37228691)
Rust's slog ( https://docs.rs/slog/ ) does:

  use slog::info;
  ...
  info!("hello, world"; "user" => os.Getenv("USER"));
Re:Structured Logging with Slog by codetrotter on Tuesday August 22, 2023 @08:43PM (#37230504)
Yup! Love the Rust slog crate. Initially thought OP was gonna be about Rust slog, before I saw the domain mention Go.
Re:Structured Logging with Slog by masklinn on Wednesday August 23, 2023 @02:24AM (#37232453)
It can do that because Rust’s macros can have their own mini language at the top level, and can transform that into whatever data structure they want under the cover.

For better or for worse, Go doesn’t have that.

Re:Structured Logging with Slog by masklinn on Wednesday August 23, 2023 @05:17AM (#37233551)
Also funny story: in Perl “,” can be spelled “=>” specifically for this sort of use cases, when you write

    my %hash = (
        Foo => “bar”,
        Baz => “qux”,
    );
the behaviour is no different than

    my %hash = (
        Foo, “bar”,
        Baz, “qux”,
    );
it’s just that a literal plist in a hash context is interpreted as a hash.

Hence “=>” being called “fat comma” in perl.

Which means you can writer

    my %h = (a => b => c => d);
or

    my %h = (a, b => c, d);

they all mean the same thing.
Re:Structured Logging with Slog by richardwhiuk on Tuesday August 29, 2023 @10:07AM (#37308183)
That all sounds questionable.
Re:Structured Logging with Slog by mlhpdx on Tuesday August 22, 2023 @07:30PM (#37229938)
Indeed. Logging a map-like object seems like a pretty basic expectation. Separate keys and values as parameters seem very C-like.
Re:Structured Logging with Slog by yencabulator on Tuesday August 22, 2023 @07:47PM (#37230075)
Personally I only intend to ever use LogAttrs/AddAttrs/GroupValue, for this reason. I really don't want to count evens and odds.
Re:Structured Logging with Slog by AkshitGarg on Wednesday August 23, 2023 @02:11AM (#37232383)
You use attrs [1] instead. I personally find this much more readable, and prefer this.

  slog.Info("hello, world", slog.String("user", os.Getenv("USER")))
[1]: https://pkg.go.dev/log/slog#hdr-Attrs_and_Values