News for hackers. Stuff that matters.
Re:A Review of the Zig Programming Language (Using Advent of... by llimllib on Monday December 27, 2021 @11:36AM (#29703902)
I found the standard library's support for strings was plenty fine, doing AoC problems in zig tests it out thoroughly. Tokenize[1], split[2] and trim[3] were the most common ones I used.

Was there something in particular you were looking for and didn't find?

[1]: https://github.com/ziglang/zig/blob/master/lib/std/mem.zig#L...

[2]: https://github.com/ziglang/zig/blob/master/lib/std/mem.zig#L...

[3]: https://github.com/ziglang/zig/blob/master/lib/std/mem.zig#L...

* After I read my own comment, I'd note that AoC tests out string manipulation pretty thoroughly but things like unicode handling not at all, so []const u8 as a string may be more annoying in the real world than in AoC answers and I haven't used zig's unicode facilities at all

Re:A Review of the Zig Programming Language (Using Advent of... by tialaramex on Monday December 27, 2021 @11:47AM (#29704030)
In AoC it's completely fine to conflate a character and a byte. Neither your daily input nor the provided tests will have anything beyond ASCII.

Which is fine for AoC, good choice, but it means the language needn't get this right, or even provide any help to programmers who need to get it right, in a similar way to how "big" numeric answers in AoC will fit in a 64-bit signed integer, never testing whether your chosen language can do better if the need arises.

Re:A Review of the Zig Programming Language (Using Advent of... by shakow on Monday December 27, 2021 @11:55AM (#29704133)
> Was there something in particular you were looking for and didn't find?

Unicode handling. Treating a string as a byte array is all fine and dandy if you're only processing english latin alphabet data, but it's a PITA as soon as you start using e.g. extended characters (math symbols, fancy quotes, ...) other languages, or emojis.

Re:A Review of the Zig Programming Language (Using Advent of... by llimllib on Monday December 27, 2021 @12:00PM (#29704188)
There is the `std.unicode` module[1] which provides standard unicode functions (encode, decode, length, iteration over code points), so I don't think it's fair to say that the language's library lacks strings in any real sense.

I will re-emphasize that I've not used it, so I cannot speak for its quality.

[1]: https://github.com/ziglang/zig/blob/master/lib/std/unicode.z...

Re:A Review of the Zig Programming Language (Using Advent of... by kristoff_it on Monday December 27, 2021 @12:38PM (#29704626)
Also here is a blog post that gives more info on how to deal with unicode in Zig.

https://zig.news/dude_the_builder/unicode-basics-in-zig-dj3

Re:A Review of the Zig Programming Language (Using Advent of... by chubot on Monday December 27, 2021 @12:21PM (#29704418)
What do you need to do with them? All my data is UTF-8, and low level code is generally parsing, which doesn't involve any of those characters. It generally just works with all special characters (e.g. on my blog).

I think Unicode on the server or CLI is very different than Unicode on the desktop/GUIs.

Since Zig interfaces well with C, it should be set up well for the harder desktop case, because the "real" Unicode libraries handling all the corner cases are written in C (or have C interfaces). I don't think even Python's unicode support is up to the task for desktop apps.

Re:A Review of the Zig Programming Language (Using Advent of... by dmitriid on Monday December 27, 2021 @05:03PM (#29707366)
> What do you need to do with them? All my data is UTF-8, and low level code is generally parsing, which doesn't involve any of those characters.

For example, file names on MacOS are Unicode. Depending on how low level your code is and what parsing it does, you will run into issues because UTF-8 will not save you.

> I think Unicode on the server or CLI is very different than Unicode on the desktop/GUIs.

This is a non-sensical statement. It's the same Unicode. Yes, you won't have all the same use cases as a GUI, but Unicode is the same.

The best example I have for when things are not handled is not strictly Unicode-related. WiFi SSIDs are 32 octets. That is any byte value at all. Recent standard modifications allow devices to specify whether the SSID is in UTF-8.

And yet. Too many devices assume that, basically, " All my data is UTF-8, and low level code is generally parsing, which doesn't involve any of those characters", and Internet is full of people asking questions like "cannot connect to WiFi with non-English SSID"

Re:A Review of the Zig Programming Language (Using Advent of... by chubot on Tuesday December 28, 2021 @11:31AM (#29715199)
Yeah filenames on OS X are a pain.

However, Python 3 has the opposite problem! All the file system APIs were changed to assume that filenames are strings (sequences of code points), when they're actually BYTES in POSIX! (which may or may not be utf-8 encoded, but commonly are)

So my point is: when you ask for "unicode" support in a language, you have to be careful what you ask for, or it can create more problems. Having less in the core language is good.

For the specific case of filenames, I think you just need basename() and dirname() routines that find '/' (or \) regardless of encoding. You don't need a separate string type.

I should have said "localization on the server" vs. "localization on desktop/GUIs". What people really want is localized apps, not just unicode, and having a string type in a language doesn't get you that far. Because localization is inherently tied to the operating system, not the language. The filename example shows that, and there are also several other issues like fonts, left-to-right, etc.

Re:A Review of the Zig Programming Language (Using Advent of... by anonymoushn on Tuesday December 28, 2021 @07:02AM (#29712924)
Woe be to they who attempt to save Japanese mp3 files on a Linux NAS and read them from a Mac.
Re:A Review of the Zig Programming Language (Using Advent of... by chubot on Tuesday December 28, 2021 @11:52AM (#29715435)
Yeah that's true, but this has nothing to do with the programming language, and everything to do with the file system implementation in the kernel, and the OS APIs.

The programming language can only do so much about this problem, and doing too much is harmful (e.g. Python 3).

The default OS X file system isn't POSIX compliant, which creates a portability issue. In POSIX file systems, filenames are bytes, and file contents are bytes. But that isn't true on OS X.

Re:A Review of the Zig Programming Language (Using Advent of... by anonymoushn on Tuesday December 28, 2021 @01:04PM (#29716169)
Agreed all around. "Strings are byte arrays" is imo the right approach to living in a hellworld where programs like filesystems expect you to send them all sorts of different precise arrangements of bytes.
Re:A Review of the Zig Programming Language (Using Advent of... by e12e on Monday December 27, 2021 @07:25PM (#29708703)
> Was there something in particular you were looking for and didn't find?

Mostly dealing with allocation - ie dynamic string handling, reading strings from a file, passing a string to a function, and returning a different string to another function and so on.

Ed: this was pretty much before solving concrete aoc problems, just figuring out how to read "suffix" from standard input, passing it to a function "prefix_suffix", getting a string "prefix suffix" back and outputting that string; trivial manipulations, but lots of dynamic allocation.