News for hackers. Stuff that matters.
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by AndyKelley on Monday December 13, 2021 @05:13AM (#29538297)
Some context:

In master branch of Zig right now (before https://github.com/ziglang/zig/pull/10330 is merged), status quo:

    $ cat ../lib/libc/glibc/*.txt | wc -c
    205219
    $ cat ../lib/libc/glibc/*.txt | xz | wc -c
    22976
Zig supports targeting every version of glibc for any target architecture. The information required to do this takes up 200 KB installation size / 22 KB tarball size, and it has the following problems (which are causing several bugs):

* fails to represent symbols that migrate from one library to another between glibc versions

* fails to represent a distinction between functions and objects

* fails to represent object sizes

With this new glibc-abi-tool:

    $ cat abilists | wc -c
    169421
    $ cat abilists | xz | wc -c
    24880
165 KB installation size / 24 KB tarball size, and all the above issues are solved. It also should be slightly faster to load/parse for the compiler since it is fewer KB to load from disk.

The code that uses this file is here: https://github.com/ziglang/zig/blob/0d7331b6f6af85bd45829514... It creates c.s, pthread.s, dl.s, etc., which are assembled into libc.so, libpthread.so, libdl.so, etc., which are placed on the linker line when cross compiling. Thanks to this improved dataset, they can now accurately model the glibc that will be on any CPU architecture, any glibc version.

If I just naively shipped every version of the .abilist files:

    $ cat (find glibc/ -name "*.abilist") | wc -c
    37041906
    $ cat (find glibc/ -name "*.abilist") | xz | wc -c
    205036
...it would be 35 MiB installation size, 200 KB tarball size. So I have effectively achieved a compression ratio of 219:1 by implementing a bespoke encoding of this information.
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by formerly_proven on Monday December 13, 2021 @08:40AM (#29539471)
Totally off-topic: I installed Zig on Saturday afternoon and by the evening I had my first program do some useful stuff, on Sunday I added some C code, then added some tests and by Sunday evening I had a pure Zig project again (and afterwards learned that translate-to-zig is a thing). I gotta say I'm very impressed with Zig and partial evaluation is just such a nice, mentally simple technique for generics and lots of other things. The syntax is pretty nice, too, though the prefix-[] are kinda weird (just way too used to C/C++'s postfix array notation) and ranged loops are somewhat awkward. The @ naming for builtins confused me for a bit because I thought the @ itself was syntax. And array literals are kinda weird sometimes though I've seen that apparently in 0.9 "&[_][]const u8{ "-std=c99", "-O3" }" can just be written as ".{ "-std=c99", "-O3" }". And I was kinda surprised about array types in function signatures being implicitly const, I think they're like that to make array types behave mostly like value types while only passing a pointer?
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by dnautics on Monday December 13, 2021 @10:55AM (#29540699)
> the prefix-[] are kinda weird (just way too used to C/C++'s postfix array notation)

I think over time the C/C++ way is going to go the way of the dodo; IIRC, rust and go both use the prefix-[]. It creates a clear, unambiguous way of defining the type (no "spiral typing" jokes for the modern languages).

Re:Show HN: A 166 KB file for cross compiling glibc for any ... by nyanpasu64 on Monday December 13, 2021 @11:24AM (#29541014)
I wish Rust used prefix &[]i32 for a slice, and [5][2]i32 for a 5x2 array. Instead it uses &[i32] for a slice, and [[i32; 2]; 5] for a 5x2 array. Reading types backwards is alive and well.
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by mmastrac on Monday December 13, 2021 @01:39PM (#29542765)
I prefer the unambiguous nesting of Rust's style. I can never remember the array shape when array bounds are listed in either prefix or postfix style. It's a tiny bit of extra typing that ensures that my understanding of the code is always 100% correct.
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by nyanpasu64 on Monday December 13, 2021 @10:38PM (#29547930)
I find Rust's array style more confusing than prefix, since prefix matches the order you index into the arrays (arr: [5][2]i32 can be validly indexed as arr[<5][<2]), and Rust's style is written backwards from how you index the array. Can you clarify how you find Rust unambiguous?
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by mmastrac on Tuesday December 14, 2021 @11:39AM (#29553603)
It's just obvious to me that it works the correct way by way of elimination of alternative parsing. The example below can be unwrapped mentally and unambiguously as "an array of 5 items of (an array of 2 items of i32)". Indexing into the array is not backwards: the outer array is deref'd first, giving you one of those five items which has length two.

  fn main() {
    let mut a: [[i32; 2]; 5] = [[0,0],[0,0],[0,0],[0,0],[0,0]];
    println!("{:?}", a[0]);
  }
Note that the Rust array declarations are also homologous to other types in the ecosystem, such as nested Option types or nested collections (eg: Option<Option<T>> or List<List<T>>).

Contrast w/the C version, where it's obvious to a skilled C programmer but you need to remember to read the bounds from right to left to determine the memory layout (ie: an array of 5 arrays of two elements each):

  int main() {
    int i[5][2] = {{1,2},{3,4}, /*...*/};
    printf("%ld\n", sizeof(i[0])/sizeof(int));
    return 0;
  }
In this case you are required to mentally combine the type on the left of the variable with the bounds on the right of the variable to parse a type declaration in your head that eventually looks like the Rust version:

  (int[2])[5]
I've been doing a lot of professional C work lately so it's currently "swapped in", but I find that any time I pick C up after spending time away I find it ambiguous and I have to reason through the ordering of the declarations.

Obviously this is all IMHO and YMMV, but this developer does find it significantly easier to unambiguously parse Rust types.

Re:Show HN: A 166 KB file for cross compiling glibc for any ... by nyanpasu64 on Tuesday December 14, 2021 @06:32PM (#29559627)
> Note that the Rust array declarations are also homologous to other types in the ecosystem, such as nested Option types or nested collections (eg: Option<Option<T>> or List<List<T>>).

To me, [[i32; 1]; 10] is homologous to Items<Items<T, ZeroOrOne>, Unlimited>, not Vec<Option<i32>>. To me, it make more sense to put the size before the inner type (either [1; i32] or [1]i32), since you need to index the array before getting an instance of the type.

Re:Show HN: A 166 KB file for cross compiling glibc for any ... by dnautics on Monday December 13, 2021 @01:02PM (#29542279)
eh, the type surrounded by the [] "operator" is equally unambiguous as prefix-[], so it's not the worst.
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by Spex_guy on Monday December 13, 2021 @01:02PM (#29542274)
Your instinct is right about array types, semantically in Zig they are passed by value. Parameter values are always const though, which allows the compiler to sometimes use pass-by-const-ref to avoid copying large pieces of data. Since arrays are proper value types, we can also make strongly typed pointers to them, like `*[3][4]u32`.
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by gavinray on Monday December 13, 2021 @02:19PM (#29543253)
Can I use this incredible feature with other toolchains, or existing artifacts that have a glibc dependency?

Say if I am using a language/toolchain which pulls in a C/C++ compiler, are you able to substitute "zig cc" there and explicitly cross-compile to a different GLIBC version?

Re:Show HN: A 166 KB file for cross compiling glibc for any ... by AndyKelley on Monday December 13, 2021 @02:21PM (#29543280)
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by gavinray on Monday December 13, 2021 @02:23PM (#29543295)
Incredible, thanks a ton for doing this work!
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by lhorie on Monday December 13, 2021 @12:30PM (#29541871)
How much work is it to then maintain this bespoke encoding? Without having much context on it, it sounds similar to having to maintain unicode identifier character tables for unicode-aware parsers (e.g. Javascript's). Updating unicode tables after unicode version releases can be done w/ somewhat hack-ish scripting of parsing txt spec files but it's admittedly not super clean, I'm curious what's involved when dealing w/ abi lists over time.
Re:Show HN: A 166 KB file for cross compiling glibc for any ... by Spex_guy on Monday December 13, 2021 @12:59PM (#29542230)
The process of adding new versions of glibc to the data set is described in the readme. If something changes in a way the format doesn't handle, we can just change it and update the code in the compiler at the same time. The file ships as part of the compiler distribution so we don't have to worry about backwards compatibility.