ahgamut

    wget https://cosmo.zip/pub/cosmos/bin/python -qO python.com
    chmod +x python.com
    ./python.com

    ./python.com -m pip download Click
    mkdir -p Lib && cd Lib
    unzip ../click*.whl
    cd ..
    zip -qr ./python.com Lib/
    ./python.com # can now import click

[1]: https://github.com/jart/cosmopolitan/commit/4e9566cd3328626d...

  python -m nuitka example.py --no-compile

From what I understand, this appears to a be separate binary from GCC/Clang that does static analysis and outputs C99.

Can this be a GCC plugin? I know we can write plugins that are activated when a specific macro is provided, and the GCC plugin event list allows intercepting the AST at every function declaration/definition. Unless you're rewriting the AST substantially, I feel this could be a compiler plugin. I'd like to know a bit more about what kinds of AST transformations/checks are run as part of Cake.

I'd like to see this evidence, and by that I don't mean someone just writing a blog post or tweeting "hey I asked an LLM to do this, and wow". Is there a numerical measurement, like training loss or perplexity, that quantifies "outside the training set"? Otherwise, I find it difficult to take statements like the above seriously.

LLMs can do some interesting things with text, no doubt. But these models are trained on terabytes of data. Can you really guarantee "there is no part of my query that is in the training set, not even reworded"? Perhaps we can grep through the training set every time one of these claims are made.

My above repo contains example with the Rust standard library that build as fat executables with Cosmopolitan Libc.

I also got ripgrep to build https://github.com/ahgamut/ripgrep/tree/cosmopolitan, but it wasn't part of the cosmocc binaries because some crates that are part of the build require `#!feature(rustc_private)]` to be added to their source code. Same goes for bat and fd.

To summarize, Rust CLI tools can be built with Cosmopolitan Libc, but to build them like we can build gcc/emacs/git right now, we will need some support and Rust expertise.

pip install requires modifying the APE, so I end up installing pure-Python libraries as follows:

    mkdir Lib
    ./python -m pip download httpx
    unzip sqlite*.whl -d ./Lib
    mv python python.com # in case the next step doesn't work
    zip -r ./python.com Lib
    mv python.com python

Also, to add new pure-python packages, you just need to unzip them into a local folder "./Lib", and add that folder to the APE.

I was sad to see setuptools officially deprecated, because it looks like another way in which Python packaging is being red-taped away for a non-expert. If someone like the OP (who has 10+ years programming Python) had to do so much for what appears to be a zstd CFFI/Rust wrapper, where does that leave the rest of us?

Here's a python package of mine that uses setup.py: https://github.com/ahgamut/cliquematch/blob/master/setup.py which I have not upgraded to the new tool(s) yet. I think I will need to upgrade it soon. If anyone has suggestions for a tool that will _fully replace_ setup.py, I would like to see tutorials with the following examples:

1. How would I build a package that has pure-Python files and data files? With setuptools I would use maybe MANIFEST.in or package_dir.

2. How would I build a package that has a CPython extension accessed via cffi? (this post points to the answer)

3. How would I build a package that has a CPython extension _without_ cffi, that just wraps some small C code I wrote with CPython's API? What about an extension that uses PyBind11? What about an extension that uses Rust?

4. How would I build a package that requires a "system" package like libblas-dev? Can something like numpy be built optimally without ever writing setup.py? What would a config for that look like? Last I remember numpy used their own patch of distutils to build, I wonder what it is now.

   I believe the problem is that bridge in the US (not true in some other countries) lacks a "critical mass" of 20 and 30 somethings.

- How much does performance improve due to this No-GIL thing? Is it greater than 2x? For what workloads? - Do I have to compile two versions of every extension (gil/nogil)? I would prefer building extensions does not get any more complicated. - Can I automatically update my code to handle nogil? (a tool like lib2to3 or six)

Would you consider a code pattern like that to be common? What other codebases apart from busybox have it? If there are many examples, I might spend some time trying to update my patch to handle patterns of that kind.

Look at the list of constants here: https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c...

Every time I used any of these constants, I'd have to load a whole bunch of them into my binary as a large lookup table, and go through that table every time I needed a check in my program. It might not be that slow, but I believe it would definitely be noticeable.

My goal was to make porting easier without changing a lot of source code in either the libc or in the software I was trying to port, and still produce binaries that are close or better in performance. Under those constraints, this gcc patch seemed like the best way to simplify the process.

If I run into enough codebases where SIGHUP is used as an array index initializer, I will probably attempt your suggestion just to measure the tradeoffs. Or you could try it out and let me know if a separate set of constants is better.

I specifically wanted to avoid rewriting the world in order to port established codebases like git/curl to Cosmopolitan Libc.

My goal with this gcc patch was to answer the question: "what is the minimum amount of source code I would have to manually change in order to port software to Cosmopolitan Libc?". As we find out, sometimes we don't have to change anything to compile code for an Actually Portable Executable -- just do the usual ./configure && make, and a small `objcopy` command at the end to create the APE.

IIRC over the last year, I've only commented once on lobste.rs, when it came to something about Python's import system, and only upvoted my own blog posts about Cosmopolitan Libc and related comments + maybe one or two other posts. I disagree with the ban, but I can see why my account would get banned as a spammer (one of the accounts banned alongside mine had a double account I think).

lobste.rs is a wonderful community, and I'm happy to see that someone there atleast mentioned this blog post. Hopefully I can get un-banned at some point.

Luckily I can still comment on HN though!

I agree with you though -- an extensive testing setup will reveal more things for improvement, be it in my patch or in the libc itself. At the moment I only have direct access to Debian 11/Fedora 35/FreeBSD 13, but I expect we will soon find a nice way to run tests across the different operating systems automatically. One of my ideas was to setup a separate drive or folder with all the executables, accessible from different operating systems/VMs, and then run the tests from each. What do you think?

In the meantime, the Cosmopolitan Libc monorepo contains binaries of gcc-11 and binutils that have been built with my patch, you can use those for now :)

This gcc patch makes such build scripts simpler, because you will need to change less of your code -- let me know how it works!

Also got ripgrep to work: https://github.com/ahgamut/ripgrep/tree/cosmopolitan

Might need to update the build scripts a little to handle the latest updates in Rust and Cosmo (we'll need gcc-11 now), but I expect to get Rust working again soon enough.

I too thought it was a typo when I caused it to occur while building BLIS, but then I found out it was because I had added some AVX thing to the config that my local computer did not have.

The default build parameters in busybox make a lot of OS-related assumptions, so I'd recommend you use make menuconfig instead of make depconfig. in the TUI, you can disable all console and networking utilities, linux modprobe utilities, some linux system utilities, free, uptime, etc. Start with a minimal config and then slowly add things.

In terms of source code changes: modify `u_signal_names.c` to use a switch statement instead of the amazing "use SIGHUP as an array index" method it follows, use int32_t instead of the "smallint" type that busybox seems to prefer, and IIRC there's an enum in there somewhere that should be rewritten as a #define.

That should get you pretty close to a successful build.

[1]: https://github.com/ahgamut/gcc/tree/portcosmo-11.2

The superconfigure script is just a wrapper around the usual configure script used to build your software, supplying flags like --enable-static.

With my gcc patch, you can now build software like vim, emacs, ninja, bash, git, gcc etc with Cosmopolitan Libc, via their usual autotools/cmake-style build system. The built executables should run on Linux, FreeBSD, MacOS, OpenBSD, NetBSD, and also Windows (although I haven't tested Windows yet.)

Here's a list of software I got to build with this technique: https://github.com/ahgamut/superconfigure

The superconfigure script is just a wrapper around the usual configure script used to build your software, supplying flags like --enable-static.

If you want to build gcc using Cosmopolitan Libc -- try out this repo: https://github.com/ahgamut/musl-cross-make/tree/gccbuild