News for hackers. Stuff that matters.
Re:Python 3.11.0 final is now available by coldtea on Tuesday October 25, 2022 @06:29AM (#33328618)
That the stdlib added support for parsing TOML but not for serializing to it (at least that was the case I read about the 3.11 when it was upcoming), sounds really short-sighted - couldn't they just add that too?
Re:Python 3.11.0 final is now available by iainmerrick on Tuesday October 25, 2022 @06:39AM (#33328708)
Parsing TOML is clearly useful, but the case for serialization is much less clear.

It's meant to be human-writable, and offers multiple ways to express the same table data; what format should a TOML serializer use?

I can see that you might want to programmatically edit a TOML file, preserving the rest of its layout unchanged. That's a bit fiddly to get right, and needs a different interface from a pure serializer. If the correct design isn't obvious, better to leave it out for now. It could still be added later.

Re:Python 3.11.0 final is now available by coldtea on Tuesday October 25, 2022 @06:54AM (#33328827)
>Parsing TOML is clearly useful, but the case for serialization is much less clear.

I never met any format where parsing it is useful but serializing to it is not. Maybe some binary format where I only care for e.g. playing a video or decoding an image. For a configuration format though?

The obvious use case is reading, altering AND writing back.

>It's meant to be human-writable, and offers multiple ways to express the same table data; what format should a TOML serializer use?

It should just pick one and stick with it?

If N versions of the format express the same data (deserialize into the same structure) then it doesn't really matter from a functional way which they pick.

Users would still need to pick a TOML lib to write data - so they already OK with that lib picking a specific way. Why wouldn't they be OK with Python do it?

If they don't want nobody to mess with their hand-written TOML files, they can always just not write them back from Python.

Re:Python 3.11.0 final is now available by iainmerrick on Tuesday October 25, 2022 @08:00AM (#33329354)
If you don't care about the format, why would you be using TOML in the first place? Just use JSON.
Re:Python 3.11.0 final is now available by coldtea on Tuesday October 25, 2022 @10:16AM (#33330744)
Who said I don't care about the format in general?

I don't care about specific stylistic options in serializing to the format that are all parsed the same way anyway.

Re:Python 3.11.0 final is now available by iainmerrick on Tuesday October 25, 2022 @05:01PM (#33336143)
That was how I interpreted this (but I didn’t say “in general”):

If N versions of the format express the same data (deserialize into the same structure) then it doesn't really matter from a functional way which they pick.

To clarify my point, if you’re using TOML, presumably you want to be able to format things nicely by hand because that’s the whole point of TOML. If you don’t need to do that, just use JSON. So I don’t see a whole lot of value in a tool for serialising data into boilerplate TOML (or YAML, etc).

Re:Python 3.11.0 final is now available by coldtea on Wednesday October 26, 2022 @03:33AM (#33340830)
>presumably you want to be able to format things nicely by hand because that’s the whole point of TOML.

For me the whole point of TOML is as a stricter, saner, YAML-type (human readable that is) config first, and a format used in several places, including upcoming Python standards, second.

Couldn't care less about formatting things nicely by hand.

Re:Python 3.11.0 final is now available by tarranoth on Tuesday October 25, 2022 @08:18AM (#33329495)
What about yaml? Most people use it for configuration, but few would write out a yaml file as output instead of a json file even though they could.
Re:Python 3.11.0 final is now available by gtirloni on Tuesday October 25, 2022 @09:56AM (#33330510)
If you spend a few minutes thinking about practical implications of a piece of code parsing toml and writing it back with stylistic changes, I'm sure you'll find many such situations.
Re:Python 3.11.0 final is now available by coldtea on Tuesday October 25, 2022 @10:15AM (#33330733)
I covered that though: this would be the case for ANY existing TOML serializing library. People already use those, and are OK with them picking a specific output style. So, they could use the functionality in the stdlib too.
Re:Python 3.11.0 final is now available by dragonwriter on Wednesday October 26, 2022 @11:13PM (#33352640)
> The obvious use case is reading, altering AND writing back

That’s actually (if what you are reading is human-edited input thar you might transform but expect to be human edited) a hard case, because then you want to keep as much as possible the format entered in, while accommodating changes, which is easy to subjectively evaluate but seems potebtially hard to express mechanically in a way which would provide generally optimal UX. You don’t want to hand edit config and have a UI, but have an “I touched it with the UI and made a couple small changes out of dozens of entries and now its unrecognizable”.

Re:Python 3.11.0 final is now available by hbrn on Tuesday October 25, 2022 @10:31AM (#33330957)
> If the correct design isn't obvious, better to leave it out for now

In the face of ambiguity, refuse the temptation to guess

Re:Python 3.11.0 final is now available by emblaegh on Tuesday October 25, 2022 @06:38AM (#33328700)
I assume reading toml files constitutes 90% of the use cases, as it's mostly used for configuration, much like the configparser module.
Re:Python 3.11.0 final is now available by coldtea on Tuesday October 25, 2022 @07:00AM (#33328870)
>as it's mostly used for configuration

Well, people write configuration in JSON, XML, and INI and process those and re-serialize to those formats too

It's also handy for making one-off scripts for changes to a big-ish configuration file - even if it's not a part of some automated pipeline.

Re:Python 3.11.0 final is now available by BiteCode_dev on Tuesday October 25, 2022 @06:44AM (#33328744)
There was a long debate about this, and the conclusion was:

There is one way to read toml, but a lot of ways to write it (do you preserve formatting, what do you do with comments, do you allow partial updates...). Therefor reading is easy to get right, but writing less so, and once it's in the stdlib, we can't make quick changes to the API.

Since reading is already very useful, and tomli author's is ready to provide that for free, let's include it now and see writing later.

Re:Python 3.11.0 final is now available by coldtea on Tuesday October 25, 2022 @06:53AM (#33328815)
>There is one way to read toml, but a lot of ways to write it (do you preserve formatting, what do you do with comments, do you allow partial updates..

Then don't use TOML for internal stuff?

Or just settle on a way your lib writes it, and if every parser out there can read it anyway, and get the same deserialized structure, it doesn't matter which way you write it exactly...

Re:Python 3.11.0 final is now available by BiteCode_dev on Tuesday October 25, 2022 @07:00AM (#33328867)
I'm just giving you a summary of the rational.

As for it was a good idea, this has already been debated and acted upon, there is no need to repeat exactly the same arguments.

You will find the detailed rational in the PEP, and link to the various discussions.

Re:Python 3.11.0 final is now available by coldtea on Tuesday October 25, 2022 @07:07AM (#33328910)
>As for it was a good idea, this has already been debated and acted upon, there is no need to repeat exactly the same arguments.

Well, the same debates happened over many other decisions debated, acted upon, and later regretted and reverted. At some point the GIL seemed like a good decision too!

Re:Python 3.11.0 final is now available by macintux on Tuesday October 25, 2022 @07:40AM (#33329183)
Obviously the complexity level is different, but to me you sound like someone complaining about Linux adding support for a new filesystem, but read-only at first. Why all the hostility?
Re:Python 3.11.0 final is now available by coldtea on Tuesday October 25, 2022 @08:47AM (#33329749)
It's a complain about a release decision. Where's the hostillity?

No need to project psychology on it...

Re:Python 3.11.0 final is now available by iainmerrick on Tuesday October 25, 2022 @08:04AM (#33329374)
That's an argument for not adding a TOML writer right now. The correct design isn't obvious so there'd be a risk of a bad design getting locked in.
Re:Python 3.11.0 final is now available by kortex on Tuesday October 25, 2022 @08:14AM (#33329457)
The GIL is a good decision and has been a sound engineering tradeoff for decades. It's only been the past few years that running the interpreter without a GIL has been even remotely feasible (cf the gilectomy project). The GIL is a relatively straightforward way to hold back a tremendous amount of complexity.

Similarly, TOML writing is a ton of completely. The org isn't opposed to adding toml writing to the stdlib fundamentally, they just aren't rushing and want to hammer out all of the grossness first.

Re:Python 3.11.0 final is now available by sgtnoodle on Tuesday October 25, 2022 @01:54PM (#33333978)
I have plenty of cases with yaml where I care about the formatting of the serialized output. I have to use obscure hooks into pyyaml's inner workings to tweak things just right. In some cases, I end up having to do some of the serialization myself. While technically it doesn't matter because it's all equivalent once in memory, there are externalized costs once you're integrated into a large system. The whole point of using something like yaml or toml is to be human friendly.