The performance boost in tools like ripgrep and tokei is insane compared to the tools they replace (grep and cloc respectively).
ripgrep is an excellent tool. But it's not a grep replacement. And should not ever be.
Please forgive me my ignorance but what's wrong with bash? I'm still using it on all servers and workstations, I constantly write scripts for it, some fairly complex. It's not an obsolete project and it looks like a mainstream shell for me. Am I wrong?
Update: yeah, I realize now that this was about the original Bourne Shell, not bash.
You have a point here. I have to agree.
"X but rewritten in Z" is a terrible marketing, though. Makes me instantly want to hate the tool and its authors. (Love rust. Hate the vibe).
Compare "I have typed a setuid() wrapper in rust" vs "I'm the author of sudo-rs".
How would you even tell people you made a better rust crate without using the word “rust?”
Rust folks are claiming excellent interoperability with C binaries. Why the need for a rewrite then?
Interoperability runs both ways, everyone currently taking a dependency on the C library can swap in the rust library in its place and see the same benefits
or Korn shell.
https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#can...
Those "core standards" that you talk about didn't spring fully formed from the earth. They came about from competition and beating out and replacing the old "core standards" that lots of people argued very strongly for should not ever be replaced. When I was starting out my career I was told by experienced people that I should not learn to rely on the GNU tool features, since they're far from ubiquitous and probably won't be installed on most systems I'll be working on.
Tokei _finishes_ before cloc can print its help text. I wrote this post in less time than it took `cloc .` to count all the files in my project, probably because it doesn't know to ignore `target/`.
cloc is a Perl script, so it has the interpreter startup time.
2) I think the MIT license was a mistake. These are often cloning GNU utilities, so referencing GNU source in its original language and then re-implementing it in Rust would be the obvious thing to do. But porting GPL-licensed code to an MIT licensed project is not allowed. Instead, the utilities must be re-implemented from scratch, which seems like a waste of effort. I would be interested in doing the work of porting GNU source to Rust, but I'm not interested in re-writing them all from scratch, so I haven't contributed to this project.
1 - GNU utilities is ancient crufty #IFDEF'd C that's been in maintenance mode for decades. You want code to handle quirks of Tru64 and Ultrix? You got it.
2 - Waving your hands around 'the community will take care of it' is magical thinking. C developers don't grow on trees. C tooling is kinda weird and doesn't resemble anything modern - good luck finding enough VOLUNTEER C developers to make your goals happen.
[1] https://discourse.ubuntu.com/t/adopting-sudo-rs-by-default-i...
How does this interact with dynamic linking? Doesn't the current Rust toolchain mandate static linking?
Use crate-type=["cdylib"]
Plenty do not, especially on Apple and Microsoft platforms because they always favoured other approaches to bare bones UNIX support on their dynamic linkers, and C++ compilers.
(which actually slightly pre-dates C++, I think?)
No. C++ is from 1985 (https://en.wikipedia.org/wiki/C%2B%2B), COM from 1993 (https://en.wikipedia.org/wiki/Component_Object_Model)
The sum size of a dynamic binary plus the dynamic libraries may be larger than one static linked binary, but whether that holds for more static binaries (2, 3, or 100s) depends on the surface area your application uses of those libraries. It's relatively common to see certain large libraries only dynamically linked, with the build going to great lengths to build certain libraries as shared objects with the executables linking them using a location-relative RPATH (using the $ORIGIN feature) to avoid the extra binary size bloat over large sets of binaries.
They are often conflated because you can't have shared dependencies with static linking, and bundling dynamically linked libraries is uncommon in FOSS Linux software. It's very common on Windows or with commercial software on Linux though.
[In case anybody is confused by your utterance, yes of course this works in Rust]
I eagerly await the results!
Also Go does produce fully static binaries on Linux and so it's at least reasonable to incorrectly guess that Rust does the same.
Definitely shouldn't be so confident though!
Here's nu, a shell in Rust:
$ ldd ~/.cargo/bin/nu
linux-vdso.so.1 (0x00007f473ba46000)
libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007f47398f2000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f4739200000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f473b9cd000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4739110000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4738f1a000)
/lib64/ld-linux-x86-64.so.2 (0x00007f473ba48000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f473b9ab000)
libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007f4738e50000)
$ ldd /bin/sh
linux-vdso.so.1 (0x00007f88ae6b0000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f88ae44b000)
/lib64/ld-linux-x86-64.so.2 (0x00007f88ae6b2000)The problem of increased RAM requirements and constant rebuilds are still very real, if only slightly less big because of dynamically linking C.
Your second paragraph is either a meaningless observation on the difference between static and dynamic linking or also incorrect. Not sure what your intent was.
Mac does this, and Windows pretty much does it too. There was an attempt to do this on Linux with the Linux Standard Base, but it never really worked and they gave up years ago. So on Linux if you want a truly portable application you can pretty much only rely on the system providing very old versions of glibc.
It's hardly a fair comparison with old linux distros when osx certainly will not run anything old… remember they dropped rosetta, rosetta2, 32bit support, opengl… (list continues).
And I don't think you can expect windows xp to run binaries for windows 11 either.
So I don't understand why you think this is perfectly reasonable to expect on linux, when no other OS has ever supported it.
Care to explain?
This way you can get small binaries with readable assembly.
Only relinking, which you can make cheap for your non-release builds.
Dynamic linking needs relinking everytime you run the program!
This is why Apple made such a big point of having a better ABI approach on Swift, after their experience with C++ and Objective-C.
While on Microsoft side, you will notice that all talks from Victor Ciura on Rust conferences have dealing with ABI as one of the key points Microsoft is dealing with in the context of Rust adoption.
Easy, not free. In many languages, extra work is needed to provide a C interface. Strings may have to be converted to zero terminated byte arrays, memory that can be garbage collected may have to be locked, structs may mean having to be converted to C struct layout, etc.
I don't know of a sed equivalent, but I guess that would be easy to implement as Rust has good regex support (see ripgrep), and 90%+ of sed usage is search-and-replace. The other commands don't look hard to implement and because they are not used as much, optimizing these is less of a priority.
I don't know about awk, it is a full programming language, but I guess it is far from an impossible task to implement.
Now the real hard part is making a true, bug-for-bug compatible replacement of the GNU version of these tools, but while good to have, it is not strictly necessary. For example, Busybox is very popular, maybe even more so than GNU in terms of number of devices, and it has its own (likely simplified) version of grep, sed and awk.
https://github.com/trifectatechfoundation/libbzip2-rs/blob/8...
I'm not familiar enough with the symbols of bzip2 to say anything about ABI compatibility.
I have a toy project to explore things like that, but it's difficult to set aside the amount of time needed to maintain an implementation of the GNU operating system. I would welcome pull requests though:
> [...] doesn't that mean the implementation is just... finished?
I don't think that it _necessarily_ means that, e.g. all projects that haven't had a release since 2019 aren't finished? Probably most of them are simply abandoned?
On the other hand, a finished implementation is certainly a _possible_ explanation for why there have been no releases.
In this specific case, there are a handful of open bugs on their issue tracker. So that would indicate that the project isn't finished.
ref: https://sourceware.org/bugzilla/buglist.cgi?product=bzip2
Although much code can be optimized to get it to run 10-15% faster, if that comes at the expense of legibility then such "feature" get rejected nowadays. Translating an existing codebase into a language that makes things more difficult¹ and (because of that) has (and most likely will have) fewer engineers willing to working in it looks very much akin to applying legibility-affecting optimizations to me.
¹ ...and not only. I've already described in other comments some of the issues I see with it as a software development tool, here https://news.ycombinator.com/item?id=31565024 here https://news.ycombinator.com/item?id=33390634 and here https://news.ycombinator.com/item?id=42241516
Makes sense, and I'd probably make the same call if I was a maintainer and someone submitted a patch which increased performance at the cost of maintainability...
> Translating an existing codebase into a language that makes things more difficult¹ and (because of that) has (and most likely will have) fewer engineers willing to working in it looks very much akin to applying legibility-affecting optimizations to me.
Here I have to personally disagree. I think that Rust is easier than both C and C++. Especially when coming into an already existing project.
The chance of me contributing to a Rust project is higher than to a C project, because I feel more comfortable in knowing that my code is "correct". It's also easier to match the "style" since most Rust projects follow the same structures and patterns, whereas in C it can vary wildly.
E.g. I contributed my first feature to Valkey (Redis fork, C codebase) recently, and figuring out how the tests worked took me quite some time. In fact, in the end I couldn't figure out how to run a specific test so I just ran all tests and grepped the output for my test name. But the tests take tens of minutes to run so this was sub-optimal. On the other hand, 99% of all Rust projects use `cargo test`, and to run a single test I can just click the play button in my editor (Zed) that shows up next to the test (or `cargo test "testname"`).
(with this said, I think that Valkey is a really well structured code base! Great work on their part)
Anyhow, this is just to illustrate my experience. I'm sure that for someone more used to C or C++ they would be more productive in that. And I could go on for ages on all the features that make me miss Rust every day at work when I have to work in other languages, especially algebraic data types!
Fedora recently swapped the original Adler zlib implementation with zlib-ng, so that sort of thing isn't impossible. You just need to provide a C ABI compatible with the original one.