News for hackers. Stuff that matters.
Re:Cross-platform Rust rewrite of the GNU coreutils by austinjp on Wednesday January 03, 2024 @04:41AM (#38852261)
Genuine question: since these are core utils, and probably used billions of times every day, is anyone actually going to switch to this version? I see that the intention is for this to be a drop-in replacement, but some options and behaviours are still different.

To clarify, I'm not intending this as a negative comment. It's an impressive project, and aiming for cross-platform sharing of scripts seems a worthy goal. The graph of progress against the GNU test suite is neat and encouraging. However, I can't imagine anyone in Unixy lands moving to this - although I may be wrong there - so it feels like a "compatibility" project for MacOS and Windows. I'm not familiar with the situation on MacOS but how bad is the compatibility issue? Similarly, on Windows what's the situation with WSL or VMs or even Cygwin? Is performance the issue?

Do people actually want this, is there a market for it? It's got 17.8k stars so seemingly so? Again, I'm not intending to cast aspersions, just trying to understand the audience. This is clearly different from a "for the fun of it" side project.

Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @07:47AM (#38853429)
This is “just” for changing the license of the coreutils so companies can use a compatible version and do not share the source of they get to modify them. In other words, it’s mostly for people who dislike GPL because of its virality.

More extreme people may see it as a major milestone in “killing” GPL.

Re:Cross-platform Rust rewrite of the GNU coreutils by wirrbel on Wednesday January 03, 2024 @08:13AM (#38853651)
It would definitely not be my motivation. I would not modify the gnu coreutils in many cases because they lack tests and are sometimes written in pre-ansi c.

At some point the GNU project focused their attention more on leading “gnu + Linux” debates and not so much on developing a stable and secure OS. So now it’s legacy cruft.

Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @08:26AM (#38853773)
Then, why not license uutils in GNU/GPLv2+ (or v3+) again as a spiritual successor to GNU Coreutils, but use a "permissive" license like BSD to allow "free for all" incl. but not limited to closing the source?

What happens if the uutils team says that they're not releasing the source of the latest version but the $CURRENT-5 from now on, moving to a closed source, open baggage model? What prevents them from pulling an effective EEE?

Re:Cross-platform Rust rewrite of the GNU coreutils by Sanzig on Wednesday January 03, 2024 @10:22AM (#38855103)
> What happens if the uutils team says that they're not releasing the source of the latest version but the $CURRENT-5 from now on, moving to a closed source, open baggage model? What prevents them from pulling an effective EEE?

I don't follow this argument. This can happen under GPL as well. Nothing stops the copyright holder from relicensing future versions of the software under a different license. The existing versions already out there under GPL/LPGL/MIT/BSD, there's no take backs, but the copyright holder is free to do whatever they want with future versions.

What would happen in your hypothetical scenario is that everyone would get really really angry with the uutils team and the latest open source version would get forked by the community. The proprietary one would wither and die, because who in their right mind wants a proprietary set of coreutils?

Fundamentally, the choice between copyleft and permissive is simply if you care that someone takes your software and incorporates it into a proprietary package. Clearly, the uutils team doesn't care if eg. Apple makes a proprietary fork of uutils for OS X. And that's their prerogative.

Re:Cross-platform Rust rewrite of the GNU coreutils by josephcsible on Wednesday January 03, 2024 @11:17AM (#38855823)
> Nothing stops the copyright holder from relicensing future versions of the software under a different license.

Accepting contributions without a CLA does, because then there's too many copyright holders to do so.

Re:Cross-platform Rust rewrite of the GNU coreutils by NewJazz on Wednesday January 03, 2024 @12:06PM (#38856499)
Doesn't the GNU project require copyright assignment?
Re:Cross-platform Rust rewrite of the GNU coreutils by josephcsible on Wednesday January 03, 2024 @02:41PM (#38858779)
Yes, but I trust the FSF to not do that. There's basically no other entity I'd give the same trust to, though.
Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @11:37AM (#38856092)
...and a single veto stops everything on its track (which is good).
Re:Cross-platform Rust rewrite of the GNU coreutils by wirrbel on Thursday January 04, 2024 @10:40AM (#38868275)
Armin Ronacher once put it like this (quoting from memory), why should I choose GPL if I don’t plan on enforcing copyleft?

The GPL is an attempt at software freedom by restricting freedoms to have a lever that leads to greater overall freedoms.

In some way it has worked for the Linux kernel if you look at contributions wrt drivers. But I am not sure so much that it worked so well for other aspects

Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Friday January 05, 2024 @08:18AM (#38878738)
Because first, it's self enforcing since GPL is court tested, second it's also a stance.

I choose GPL, because I do not code these tools with my programmer hat. I code them on my free time, primarily for myself, to be used by people who appreciate the work went into them and find these tools beneficial.

These tools, while vary in sophistication, are high quality items which are built for their users, and not open to be monetized by another company just because they can build something with or on top of it.

I have no qualms with Open Source software when done honestly. Most of today's Open Source projects are not honest.

Try to deploy a service or compile an Open Source tool solely from the provided source code. 99.99% of the time you'll wish that you land flat on your face, which would be easier and less painful.

I choose GPL, because not only I promise that you'll be able to build the thing I released, I promise that I'll make it buildable with minimum fuss and effort as much as possible.

What I put out is complete opposite of a run of the mill Open Source Software. Free, easy to understand, easy to build, no moats whatsoever. It's a gift instead of a window dressing. It's a free offering with no strings attached instead of "fix our code, so you might get internet cookie points in return". It's crafted instead of produced.

Re:Cross-platform Rust rewrite of the GNU coreutils by hawski on Wednesday January 03, 2024 @08:17AM (#38853680)
Toybox has this for a goal. This project I would think is about the blessed safety of Rust.
Re:Cross-platform Rust rewrite of the GNU coreutils by giancarlostoro on Wednesday January 03, 2024 @10:31AM (#38855215)
It might be the motivation for companies like Apple to use it instead of the GNU utils, but they're still using the old GNU utils (if they still even bother putting it on Apple Silicon versions of macOS).
Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @11:30AM (#38856001)
Apple is not against GPLv2 since it doesn't prevent TiVoization, but GPLv3 is a complete no no for Apple as far as I can see.
Re:Cross-platform Rust rewrite of the GNU coreutils by tcmart14 on Wednesday January 03, 2024 @01:29PM (#38857712)
I may be wrong here, but my understanding is they don't use GNU utils, at least not any more. They use old FreeBSD utils, which is permissively licensed.
Re:Cross-platform Rust rewrite of the GNU coreutils by hollerith on Wednesday January 03, 2024 @01:32PM (#38857759)
You are not wrong. (Apple never shipped GNU coreutils or GNU tar. It did ship GCC long ago and maybe related packages like binutils.)
Re:Cross-platform Rust rewrite of the GNU coreutils by yyyk on Wednesday January 03, 2024 @10:41AM (#38855321)
There are already mostly compatible utils in FreeBSD*, as part of their own attempt to pivot from GPL to BSD following GPLv3 (I think it's understandable that an explicitly BSD-aligned project would seek to avoid GPL). So anyone who wants to avoid coreutils could already do it.

* With the exception of diff3.

https://wiki.freebsd.org/GPLinBase

Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @11:35AM (#38856062)
I don't expect a BSD to host GPLd software, but these projects are built with the spirit of open source, so they don't pull shenanigans.

The problem with permissive licenses started in my eyes with companies abusing the freedom they provide by creating *ium projects and started to push closed source applications with pseudo-open-source *ium projects. Also the same companies get these software fork it, add a small thanks with a 3pt font, printed to the space between the last page and back cover, and be done with it.

This is not illegal, but against the spirit of open source in general, and harmful on the long run.

As someone said: Open Source is about developer freedom, Open Source is about user freedom.

And developers are users of the software they don't develop, and they're cutting the branches they're sitting on.

Re:Cross-platform Rust rewrite of the GNU coreutils by wredue on Wednesday January 03, 2024 @11:00AM (#38855564)
What is viral about using a GPLd binary executable that doesn’t link?

On the whole, I am firmly against Open Source and would generally go with Source Available, but this seems like FUD.

Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @11:29AM (#38855988)
Note: Assuming that you're shipping a product which contains coreutils, regardless of it's modified or not.

First, you need to ship the source of that GPLd binary with the binary itself. Second, no derivative of that source code (and the binary as a result) can have a different license unless you're the copyright holder on that source code.

On the whole, I am firmly against Open Source and always go with Free Software, but this seems like incomplete knowledge.

edit: Added the "Note" section in the beginning. There was confusion it seems.

Re:Cross-platform Rust rewrite of the GNU coreutils by russdill on Wednesday January 03, 2024 @12:12PM (#38856574)
You absolutely do not need to ship the source code with the binary. The source offer is sufficient and the most common way of meeting the license obligation.
Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @12:29PM (#38856813)
> The source offer is sufficient and the most common way of meeting the license obligation.

Yes, you're right. And you play e-mail ping-pong while trying to find the correct person in most cases. IIRC Linksys got into "warmer than usual" water because of not honoring the offer.

Re:Cross-platform Rust rewrite of the GNU coreutils by wredue on Wednesday January 03, 2024 @11:39AM (#38856130)
Sorry dude, but simply using pre-compiled coreutils in your org doesn’t force you to ship the code, nor does it pose any licensing issues.

If you link, or modify the code, then you have some license issues. FUD.

Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @11:46AM (#38856234)
Sorry dude, but assuming that you ship a product using a precompiled GNU coreutils, you're operating under GPL.

The license states the following[0]:

3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following:

    a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,

    b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, 

    c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) 
If you're using it under your organization, you don't need to ship anything, but if you're shipping something with it, then you must.

The same applies if you're hosting a SaaS with AGPL license.

Sorry, but I'm too old for this FUD thing. I neither have the motivation, nor the desire, nor the reason to do it.

[1]: https://www.gnu.org/licenses/old-licenses/gpl-2.0.txt

Re:Cross-platform Rust rewrite of the GNU coreutils by russdill on Wednesday January 03, 2024 @12:14PM (#38856609)
He said if you use it within your org, not if you ship it. Hell, you can even modify coreutils to whatever the hell you want. So long as you aren't distributing the binary, you are under no obligation to provide source or a source offer.
Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @12:28PM (#38856790)
I think I have addressed that in the comment you just answered. There was a misunderstanding, and I tried to fix it.

I again edited the comment which lead to the confusion.

Re:Cross-platform Rust rewrite of the GNU coreutils by russdill on Wednesday January 03, 2024 @11:41AM (#38856151)
No, the majority of companies shipping a product like coreutils don't care if it's GPL. They are extremely unlikely to modify it and meeting the licensing requirements with a source offer is not burdensome. The binaries aren't linked against so there isn't some viral nature.

The issue is with GPLv3.

Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @11:55AM (#38856353)
GPLv3's biggest features are preventing source code from bricking by not running on the target hardware unless it's signed with a confidential private key, and automatically granting any patents which may encumber the source code.

So, companies doesn't like to share their source code in a usable form if they're not against GPLv2 but are against GPLv3.

Companies are being companies. It's not about the license, but allowing others to use their code on their products. So they're after smoke and mirrors, and when their mirrors are taken away, they have moved to mirrors that work.

Am I understanding that right?

Re:Cross-platform Rust rewrite of the GNU coreutils by russdill on Wednesday January 03, 2024 @12:09PM (#38856545)
Previous open source licenses focused on making sure that improvements to open source software were shared. There were created at a time when appliance and IoT like devices did not exist. Many companies have been very supportive of the model, joined open source groups, and even taken an upstream first approach to open source projects. Open source projects provide an enormous asset to companies, and it's often in their best interest to see those projects thrive.

But yes, companies are much less interested in people modifying software on devices they sell. It isn't smoke and mirrors, there are just different reasons that different individuals support open source software. And there's different licenses to suit different purposes.

Re:Cross-platform Rust rewrite of the GNU coreutils by dspillett on Wednesday January 03, 2024 @05:31AM (#38852528)
> so it feels like a "compatibility" project for MacOS and Windows

I get a similar feeling. This is more attractive than ports like cygwin as a solution because if you have this in every place you can be more sure your environments match. With ports there will be more delay getting changes/fixes in different environments, at least in theory, than with a solution that is cross-platform as a core goal. The other main options, running Linux in a VM (directly or via WSL2) adds an extra layer of friction.

A cross-platform solution is more likely to match bug-for-bug in different environments, which can be as important as matching feature-for-feature: if something is going to fail out there it will fail the same way in your dev/test environments.

Also some will want to use it from a licensing PoV. Many¹ find GPL related licences bothersome and use the GNU coreutils because there isn't (until this matches their requirements) an alternative. Similarly, the language might be an attraction to some, either ideologically or because they might want to dig into the source, though this is more of a factor for projects developing new features rather than trying to be drop-in replacements.

I expect the attraction to be relatively niche though, people won't start using it as a drop-in for the GNU tools en-mass until, for instance, popular distributions use it (or a niche distribution using it becomes more popular for some other reason).

----

[1] I am not one of that many, but they are many so this is a notable consideration.

Re:Cross-platform Rust rewrite of the GNU coreutils by cdogl on Wednesday January 03, 2024 @07:06AM (#38853135)
I mostly agree with your comment, but the source code of of many GNU coreutils is quite gnarly. It's ancient code (from my reference point as a 35yo) developed at a time when coding style was different and a much smaller community maintained it.

I think it's important for free software that people coming into the community are enthusiastic to maintain it. It took the wind out of my sails a little when I realised the GNU code base, while it produces critical tools I use every day, is written in a way that I found extremely (unnecessarily) terse and "clever". Tracing how different combinations of flags are handled is not much fun. Documentation is helpful for the user, less so for the tinkerer who is trying to understand the stack.

If this project manages to hit parity with GNU coreutils, and my distro(s) provide support, I'll switch to it purely on that basis.

Re:Cross-platform Rust rewrite of the GNU coreutils by dmd on Wednesday January 03, 2024 @08:39AM (#38853894)
I've interacted quite a bit with some of the authors, and when I've asked "why on earth did you do it this way" the answer is generally some form of "well, it saves nearly 6 bytes on disk in the source code! disk isn't free, son".

It's a different mindset and one that is no longer useful.

Re:Cross-platform Rust rewrite of the GNU coreutils by amluto on Wednesday January 03, 2024 @10:59AM (#38855546)
> It's a different mindset and one that is no longer useful.

I disagree. Sure, 6 bytes is approximately free in most contexts, but 100MB is not free, and 5GB is even less free.

But more importantly, writing code under constraints can force good behavior. For example, BIOS is a legacy mess but it’s a small, self-contained legacy mess that fits in a few kB. Compare to UEFI, which is unbelievably complicated and bug-ridden. A mess like UEFI could not have fit within the constraints of BIOS.

This is not to say that writing obscure code to save a couple bytes of source file size is at all worthwhile any more, but the idea that one should constrain bloat (design bloat, code bloat, executable bloat, network bloat, etc) is very much still valuable.

Re:Cross-platform Rust rewrite of the GNU coreutils by remus on Wednesday January 03, 2024 @11:49AM (#38856283)
I don't think it is so black and white. The original authors made trade offs that made sense at the time, but in the intervening period the context has changed and those trade offs don't necessarily make sense. The parents point is obviously exaggerated, but currently disk is cheap and the cost of inscrutable code, especially code that is run by millions or even billions of people, is extremely high: bugs are harder to spot and harder to fix, the barrier to entry for new maintainers is high, new features are harder to add etc.
Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @09:41AM (#38854575)
When I asked a graybeard why the variable names were so short, he said that "longer names affected compile duration very severely in the past, so this is why we used the shortest name possible".

While user facing programs are not faster by any means, computers got faster in some aspects after all.

Re:Cross-platform Rust rewrite of the GNU coreutils by dboreham on Wednesday January 03, 2024 @11:38AM (#38856120)
This is mostly wrong. Longer variable names never significantly affected the time to run a compiler. Back in the day the run time for compilers was mostly determined by the overhead to load in each overlay (the whole compiler didn't fit in memory). That said, often there was a hard limit on the length of identifiers that wasn't terribly long (e.g. 8 characters), so very verbose names were not possible. The more obvious reasons for preferring short variable names are: 1. It's more like math, and for simple code looks nicer, and the descriptive variable name mafia were still in diapers and 2. typing was slow and painful, so was editing (use an ASR-33 with 'ed' to find out how painful).

What your greybeard might have been talking about was to do with interpreted languages where the interpreter had to parse through the code in some situations when performing a jump/goto. This meant that the more characters there were between the jump origin and destination, the slower execution was. I remember this being a factor in early 8-bit BASIC implementations, leading to a desire to make programs as terse as possible to achieve fast execution. Later implementations tokenized the source before execution, avoiding that problem.

Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @12:09PM (#38856548)
No, we were talking about how compilers came in at least two disks (one for compiler and one for linker) when this came up.

He started these things with punched cards, and worked at the prominent companies of the time (like IBM and HP), and seen tons of bizarre computer architectures on the way like registerless processors.

He might be recalling a very specific example, or about a very slow system, but I remember the conversation well.

That thing has been probably sorted out by the mid of 90s, but I was just starting programming back then.

Oh, I'm a proud member of "descriptive variable name mafia", but not affiliated with the Java branch. They're a different breed. On the other hand, being written real math in C++ (differential equations), having three letter variable names for everything is equally painful. :)

Re:Cross-platform Rust rewrite of the GNU coreutils by johnisgood on Wednesday January 03, 2024 @11:15AM (#38855779)
I think it is important to strike a balance here, see Java codebases for the other extreme end of the spectrum where the variable name may not fit within 80 column width.
Re:Cross-platform Rust rewrite of the GNU coreutils by bayindirh on Wednesday January 03, 2024 @11:36AM (#38856088)
Of course. I name things as "configuration_storage", but not "configuration_factory_singleton_configuration_stub_constructor". I've been there, seen the horror.

Never again.

Re:Cross-platform Rust rewrite of the GNU coreutils by acchow on Wednesday January 03, 2024 @11:59AM (#38856402)
In 20 years with growing display sizes and resolutions, I’d have hoped 100 or 120 columns replaced 80
Re:Cross-platform Rust rewrite of the GNU coreutils by spencerchubb on Wednesday January 03, 2024 @12:06PM (#38856501)
I thought the column recommendation of 80 was about eye movement and reading speed, not display width.
Re:Cross-platform Rust rewrite of the GNU coreutils by ncallaway on Wednesday January 03, 2024 @12:36PM (#38856912)
I think it was both, and has become more the former now.

My preferred line-width for code would be 80 characters, not including indentation for the line, with a maximum of 120 characters including indentation.

Re:Cross-platform Rust rewrite of the GNU coreutils by lenkite on Friday January 05, 2024 @02:09PM (#38883268)
Apple OS programming beats name length for Java anyday but it never gets a bad rep thanks to apple's reality distortion field.

CMMetadataFormatDescriptionCreateWithMetadataFormatDescriptionAndMetadataSpecifications

Re:Cross-platform Rust rewrite of the GNU coreutils by dig1 on Wednesday January 03, 2024 @11:20AM (#38855865)
The thing is that coreutils is used everywhere - servers, desktops, and embedded devices, including 30-40 years old machines. You want to update coreutils on old SPARC or some even older mainframe? Every byte counts.

So, saving as many bytes as possible is still very relevant.

Re:Cross-platform Rust rewrite of the GNU coreutils by kstrauser on Wednesday January 03, 2024 @11:33AM (#38856043)
How often are we updating coreutils on 40 year old machines today? Are there more than a couple of hobby machines that old and still regularly updated with new packages?
Re:Cross-platform Rust rewrite of the GNU coreutils by dig1 on Wednesday January 03, 2024 @11:51AM (#38856298)
You'd be surprised how many banks, electrical grids, oil rigs, and large ships run on old hardware. If ain't broken, you don't replace it. But keeping the system up-to-date is always a benefit.
Re:Cross-platform Rust rewrite of the GNU coreutils by mschuster91 on Wednesday January 03, 2024 @12:17PM (#38856652)
> You'd be surprised how many banks, electrical grids, oil rigs, and large ships run on old hardware. If ain't broken, you don't replace it.

That attitude has got to die in an age of everything being exploited by bad actors. Anything that has any form of networking should have regular replacement at least of the control plane components budgeted in from the start.

Re:Cross-platform Rust rewrite of the GNU coreutils by elzbardico on Wednesday January 03, 2024 @12:34PM (#38856894)
Most of those really old devices are usually air-gaped from the internet for incidental reasons. Either they run on closed networks or have no networking at all.

You're not going to find a 30 years old machine that controls the pumps of nuclear reactor on the internet.

Re:Cross-platform Rust rewrite of the GNU coreutils by spoiler on Wednesday January 03, 2024 @12:48PM (#38857102)
But if these machines are so isolated, and given they've been running with the "if it ain't broken" mindset: why, and more importantly how do they even get updated? If they aren't being updated, why discuss constraints on updating them?
Re:Cross-platform Rust rewrite of the GNU coreutils by elzbardico on Wednesday January 03, 2024 @01:01PM (#38857299)
I bet they don't get updated, ever.
Re:Cross-platform Rust rewrite of the GNU coreutils by mschuster91 on Wednesday January 03, 2024 @03:07PM (#38859121)
> You're not going to find a 30 years old machine that controls the pumps of nuclear reactor on the internet.

Stuxnet would like to have a word with you. Airgap isn't enough to prevent malware from spreading.

Re:Cross-platform Rust rewrite of the GNU coreutils by lbhdc on Wednesday January 03, 2024 @12:24PM (#38856739)
On the otherhand, I think you could argue that it is something to strive for in more systems.

The longer our devices can last the less ewaste we generate. It may not be the easiest to secure, but it isn't impossible.

Re:Cross-platform Rust rewrite of the GNU coreutils by kstrauser on Wednesday January 03, 2024 @02:03PM (#38858262)
I don't buy into that here. If it's that mission critical and hard to replace, I wouldn't touch something as essential as coreutils without a gun to my head. In the very best case, the new version would work exactly like the old one so that the 40-year-old shell scripts holding the thing together would continue working as before.
Re:Cross-platform Rust rewrite of the GNU coreutils by stonogo on Wednesday January 03, 2024 @11:40AM (#38856139)
Coreutils might be everywhere, but uutils will only be wherever LLVM targets, because Rust isn't as portable as C. It's much easier to declare yourself cross-platform when you're not actually competing with real cross-platform software (like coreutils).
Re:Cross-platform Rust rewrite of the GNU coreutils by 9659 on Wednesday January 03, 2024 @11:02AM (#38855594)
except when it is.
Re:Cross-platform Rust rewrite of the GNU coreutils by stevehawk on Wednesday January 03, 2024 @02:41PM (#38858785)
You say that, but I recently ran out of disk space on my iPhone.. and the more I looked at the apps the more I realized "I bet these are all framework based apps and not native apps, and no one is tree-shaking their code or doing /any/ of the things you're supposed to in order to minimize disk usage.".. *shakes fist at clouds*
Re:Cross-platform Rust rewrite of the GNU coreutils by viraptor on Wednesday January 03, 2024 @07:37AM (#38853352)
This is true. I've tried to read some gnu code in the past (tar) and patch another (df) and... ran away instead. It's doable, but so messy I'd rather write my own very specific command, than try improving one of the old-style gnu ones.
Re:Cross-platform Rust rewrite of the GNU coreutils by Throw839 on Wednesday January 03, 2024 @07:34AM (#38853328)
This! I am horrified to touch anything old from GNU!
Re:Cross-platform Rust rewrite of the GNU coreutils by noirscape on Wednesday January 03, 2024 @05:14AM (#38852439)
Well for one, uutils is in rust. That alone can be pretty desirable for some people because that infers assumptions about memory safety, especially compared to the coreutils which are in C.

The license stuff is what people focus on, but to me that is much more interesting. C is a pretty decrepit language and while I don't care much for the "rewrite it in rust" cult, the coreutils are exactly the type of program rust excels at - a non-iterative target that doesn't change often and can be made with "best practices" in mind.

Not having to deal with the utterly inane amount of dead architectures that GNU projects inherit probably helps them too on that end.

Re:Cross-platform Rust rewrite of the GNU coreutils by lohnjemon on Wednesday January 03, 2024 @05:42AM (#38852610)
What kinds of memory safety bugs do you really care about in coreutils? Genuinely curious.

Given how mature and well defined the GNU Coreutils are, how small their scope is, how they are used, I really don't see the supposed security upside here.

There simply has to be a better reason to me, than "Rust good".

Re:Cross-platform Rust rewrite of the GNU coreutils by wongarsu on Wednesday January 03, 2024 @06:06AM (#38852779)
Some CVEs have happened in the past [1]. None of them memory issues, but a couple that seem unlikely in idiomatic rust or are much easier to prevent in rust.

Specifically, integer overflow is much easier to correctly handle in rust, making bugs like CVE-2015-4042 less likely, and correct handling of multibyte strings is basically enforced by the standard library, making issues like CVE-2015-4041 very unlikely in a rust implementation

1: https://www.cvedetails.com/vulnerability-list/vendor_id-72/p...

Re:Cross-platform Rust rewrite of the GNU coreutils by lohnjemon on Wednesday January 03, 2024 @07:11AM (#38853167)
So this one integer overflow in sort, a command which is never ran as root is an issue somehow, because it can cause a denial of service(it crashes)? Am I missing something here. Can I use this to exploit someone's machine?

I can search uutils/coreutils for "overflow" and get way more hits, I don't see how this is a rational thing to be afraid of within GNU Coreutils considering it's a collection of tools, that have been developed and maintained for decades and used by millions over that time period.

https://github.com/uutils/coreutils/issues/1420 https://github.com/uutils/coreutils/issues/886 https://github.com/uutils/coreutils/issues/5149

To be clear, I don't see any problems personally with any of these issues, they don't seem very exploitable to me.

However, I think that relying on Rust to be the bastion of safety merely because the name "Rust" is mentioned is nothing but a fallacy.

To me, logic bugs are the far more egregious category in something like coreutils. Me making the assumption, that something works the way it's documented, but doesn't can lead to horrible things down the road. Much more so, than any integer overflow crash could ever dream to.

Re:Cross-platform Rust rewrite of the GNU coreutils by _flux on Wednesday January 03, 2024 @07:18AM (#38853215)
But you still need to pay attention to use overflow checking versions of functions when doing arithmetics, because in release mode regular integers are not overflow-checked at runtime, unless you explicitly enable -C overflow-checks=true—whichin my opinion would be a good default for many non-performance-critical applications.

Arguably it's the "pay attention" part that causes the bug in the first place, so I don't think the performance-oriented default was a good pick.

The issue is mitigated a bit by the remaining runtime array bounds checks, but I must wonder if those checks could be removed by the optimizer when it believes e.g. a variable can never be below a certain value.

Re:Cross-platform Rust rewrite of the GNU coreutils by wongarsu on Wednesday January 03, 2024 @09:30AM (#38854435)
Apart from overflow checking in debug mode or with the right compile flag, rust also makes it a lot easier to do the right thing, e.g. 100u8.saturating_add(255) or even encoding it in the type system (`use std::num::Saturating; let mut x: Saturating<u8> = Saturating(128); x += 200; assert!(x == Saturating(255))`) (obviously you can also use Checking for explicit handling or Wrapping instead of Saturating). Meanwhile overflow handling in C/C++ is difficult, tedious and full of footguns caused by compiler optimizations.
Re:Cross-platform Rust rewrite of the GNU coreutils by _flux on Wednesday January 03, 2024 @10:25AM (#38855141)
It seems it's going to become easier to check them in C++26 with https://en.cppreference.com/w/cpp/numeric/add_sat and friends. Or if you want saturating integer types, you can find https://github.com/StefanHamminga/saturating (granted this does not seem maintained) or https://www.boost.org/doc/libs/master/libs/safe_numerics/doc... from boost for a checked integer type. https://github.com/mbeutel/slowmath gives you exception throwing checking.

I haven't tried that style in Rust—or in C++ for that matter—but is it truly much nicer than the options available for C++? Perhaps out-of-the-box experience is the winner there.

Re:Cross-platform Rust rewrite of the GNU coreutils by pitaj on Wednesday January 03, 2024 @12:23PM (#38856733)
There are lints you can enable to enforce this, rather than just "pay attention".
Re:Cross-platform Rust rewrite of the GNU coreutils by VBprogrammer on Wednesday January 03, 2024 @05:52AM (#38852684)
Wasn't it just a couple of years ago that we had shell shock? According to Wikipedia that bug was introduced in 1989. Of course, that wasn't a memory safety issue (and it's probably important to reiterate that memory safety doesn't mean free of bugs and / or exploits). But it does demonstrate that issues can exist for a long time in mature code without anyone noticing.
Re:Cross-platform Rust rewrite of the GNU coreutils by ReleaseCandidat on Wednesday January 03, 2024 @06:18AM (#38852842)
> Wasn't it just a couple of years ago that we had shell shock?

Yes, but Bash (or any other shell) isn't part of coreutils.

https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_com...

Re:Cross-platform Rust rewrite of the GNU coreutils by Corrado on Thursday January 04, 2024 @03:44AM (#38864640)
I think the upside is that these utils are not set in stone and never updated. Future maintainers will probably find it easier to work on a more modern codebase with more footgun protections. So, yes while the current utils is great, at some point the code will have to be modified and I would much rather modify clean Rust code than 30, 40, 50 year old super optimized C that no one truly understands anymore.
Re:Cross-platform Rust rewrite of the GNU coreutils by YoshiRulz on Wednesday January 03, 2024 @05:23AM (#38852484)
I use NixOS so it's feasible for me to use these as true drop-in replacements when they're done. And the reason I'd want to do that is for hardening, as the sibling commenter suggests.

The fact it's released under MIT instead of Apache (or GPL) does worry me though.

Re:Cross-platform Rust rewrite of the GNU coreutils by Snow_Falls on Wednesday January 03, 2024 @06:42AM (#38853002)
Yeah, I'm not a fan of the rust ecosystem trying to move everything from FSF-style free/libre licenses to permissive licenses.
Re:Cross-platform Rust rewrite of the GNU coreutils by m4rtink on Wednesday January 03, 2024 @08:39AM (#38853899)
Yeah - memory safety, why not (though its not a silver bullet) but why change te license to one that can be quite dangerous over time for something this important?
Re:Cross-platform Rust rewrite of the GNU coreutils by silon42 on Wednesday January 03, 2024 @09:02AM (#38854118)
Someone could release it with a relicense to GPL (and maybe LGPL if dynamically linked).
Re:Cross-platform Rust rewrite of the GNU coreutils by dartos on Wednesday January 03, 2024 @09:42AM (#38854591)
You can’t just release someone else’s BSD code under GPL.
Re:Cross-platform Rust rewrite of the GNU coreutils by Zambyte on Wednesday January 03, 2024 @09:53AM (#38854723)
Sure you can. As long as you follow the terms of the MIT license. That effectively nullifies the value of slapping the GPL on it, but you can do it.
Re:Cross-platform Rust rewrite of the GNU coreutils by dartos on Wednesday January 03, 2024 @03:33PM (#38859454)
Why even comment this?

It adds nothing but confusion to the conversation.

Re:Cross-platform Rust rewrite of the GNU coreutils by Zambyte on Wednesday January 03, 2024 @06:42PM (#38861315)
Because the ability to sublicense is the whole point of permissive copyright licenses. The point of confusion is saying that you can't sublicense, when you very explicitly can.

The reason why it would be useful to sublicense it as GPL would be to intermix it with changes that are GPL. The combined work would be covered by the terms of the GPL, and the original work would remain covered by the MIT license.

Re:Cross-platform Rust rewrite of the GNU coreutils by dartos on Friday January 05, 2024 @03:05AM (#38876844)
> Because the ability to sublicense is the whole point of permissive copyright licenses.

What makes you think this? The point of licenses like GPL (copyleft licenses) is to prevent sublicensing.

GPL sets very specific rules and you need to follow them. You can’t just ignore them and change the code’s license.

MIT isn’t copyleft in that sense (as you don’t _have_ to release changes you make to MIT code) but any code released based on MIT code must also include the MIT license.

You can’t just change the license all willy nilly, that would defeat the purpose.

Re:Cross-platform Rust rewrite of the GNU coreutils by Zambyte on Saturday January 06, 2024 @10:57AM (#38892443)
> What makes you think this?

The body of the license, particularly in contrast to the copyright granted automatically, and other copyright licenses. The MIT and BSD licenses are not very long and quite easy to digest. I recommend giving them a read.

> The point of licenses like GPL (copyleft licenses) is to prevent sublicensing.

The juxtaposition with the previous question makes me think you may have missed that your question was in reply to permissive licenses, or maybe you think that the GPL is a permissive license? Either way: I agree. The GPL does prevent (further) sublicensing.

> [...] but any code released based on MIT code must also include the MIT license.

Yes. That's why I said:

> That effectively nullifies the value of slapping the GPL on it, but you can do it.

In my previous comment. Because when you simply slap the GPL on some MIT code and release it like that, people can choose to use the MIT licensed code with the MIT license instead. Effectively nullifying the value of the GPL.

> You can’t just change the license all willy nilly, that would defeat the purpose.

Never meant to suggest you could.

Re:Cross-platform Rust rewrite of the GNU coreutils by fanf2 on Wednesday January 03, 2024 @10:20AM (#38855073)
A fair number of the gnu utilities were originally written as part of the 4.3BSD Net/2 effort to get rid of AT&T code. When the gnu project adopted the code it got relicensed from BSD to GPL.
Re:Cross-platform Rust rewrite of the GNU coreutils by Zambyte on Wednesday January 03, 2024 @02:29PM (#38858620)
Relicensed or sublicensed? Anything can be relicensed (by the copyright holder), because that is outside the terms of a license. Neither the terms of the BSD license nor GPL are relevant for that.
Re:Cross-platform Rust rewrite of the GNU coreutils by smt88 on Wednesday January 03, 2024 @05:18AM (#38852461)
> Differences with GNU are treated as bugs.

That tells me that "some options and behaviours are still different" is also being treated like a bug.

Even if this is only used by huge companies like Meta or for new projects, it will still justify its existence..

Re:Cross-platform Rust rewrite of the GNU coreutils by pletnes on Wednesday January 03, 2024 @05:37AM (#38852579)
Where I’m working, WSL doesn’t work (well), VMs are banned, and personal experience with cygwin is awful (that was on win7, but still). Everything that helps me develop software on windows which runs on linux (of course!) is warmly welcome.

I’m sure there are a thousand similar but different enterprise environments like this. At home / solo dev it makes no sense, you’d just install ubuntu / get a macbook and get working without any fuss.

Docker works but it makes test runs 100x slower (yes, I measured, it’s not made up).

Re:Cross-platform Rust rewrite of the GNU coreutils by wongarsu on Wednesday January 03, 2024 @06:14AM (#38852824)
Makes me wonder how docker on Windows handles file access? I know under the hood it uses WSL2, which in turn uses a HyperV VM. In WSL2 it's a big deal where you put there files. While windows can access the files in the Linux VM and the other way around, that requires communication over the hypervisor and is magnitudes slower than accessing the files managed by your own kernel (like C: on Windows and /home in the WSL VM)
Re:Cross-platform Rust rewrite of the GNU coreutils by thesnide on Wednesday January 03, 2024 @07:11AM (#38853170)
I use mingw/msys2 with great success.'

I even do some directx dev with it.

All with very portable code that can even be built in the standard github action ubuntu.

Re:Cross-platform Rust rewrite of the GNU coreutils by pletnes on Wednesday January 03, 2024 @07:26AM (#38853264)
Yes, and git bash is quite useful and built on those. In many ways the most useful alternative - it works and everyone already has it. But not a logical choice if you want to «deploy» something I guess.
Re:Cross-platform Rust rewrite of the GNU coreutils by VBprogrammer on Wednesday January 03, 2024 @05:47AM (#38852646)
If your test runs are slower by two orders of magnitude then something is going wrong.

For example, I've had problems on docker for Mac where accessing lots of (python) files in a volume mount was slow. I had to use some beta setting at the time which was far better (though I had to do a OS upgrade to use it, which also entailed some IT nonsense).

The overheads of docker are there but if it was that bad people wouldn't use it.

Re:Cross-platform Rust rewrite of the GNU coreutils by pletnes on Wednesday January 03, 2024 @07:30AM (#38853298)
File IO seems to be the problem. I’m guessing antivirus + virtualization overhead. Don’t see what I can change about that (use linux, sure).

Docker is still useful, it’s great for sharing a complicated setup across machines.

Re:Cross-platform Rust rewrite of the GNU coreutils by prewett on Wednesday January 03, 2024 @11:54AM (#38856338)
If I recall correctly, NTFS updates a file's access time (or some such) every time anything happens to it, so by default Windows' file I/O is slow. (Antivirus, of course, will not be helping that.) Once upon a time you could disable access time updating in the registry, although I assume if you had those kind of permissions you wouldn't be having this problem.
Re:Cross-platform Rust rewrite of the GNU coreutils by ParetoOptimal on Wednesday January 03, 2024 @09:34AM (#38854495)
It is that bad. I use a Linux VM with docker from OSX and coworkers use docker with OSX.

Their compiles take 2m, mine take 30s.

Re:Cross-platform Rust rewrite of the GNU coreutils by cesarb on Wednesday January 03, 2024 @06:07AM (#38852791)
> [...] VMs are banned [...] Docker works but [...]

I might be missing something, but isn't Docker on Windows or MacOS actually a VM running Linux (or, on Windows, perhaps it uses WSL2, which is also a VM running Linux)? How could you use Docker if VMs are banned?

Re:Cross-platform Rust rewrite of the GNU coreutils by master-lincoln on Wednesday January 03, 2024 @08:22AM (#38853738)
Docker on Windows can run without VM if the guest in the container is a Windows with a matching kernel and the process isolation is enabled in docker parameters.

Similar situation for MacOS docker containers on a MacOS host (see recently discussed https://news.ycombinator.com/item?id=37655477)

But for running Linux containers on MacOS or Windows I think you are right

Re:Cross-platform Rust rewrite of the GNU coreutils by tahnyall on Wednesday January 03, 2024 @11:12AM (#38855736)
> Where I’m working, WSL doesn’t work (well), VMs are banned, and personal experience with cygwin is awful

I'm no Windows fan but WSL has worked extremely well on several different machines I've worked on since 2015 and Cygwin was no WSL but I could get a lot done with it back in the day as well.

> Docker works but it makes test runs 100x slower (yes, I measured, it’s not made up).

Yeah, I don't know what you're doing that it could be 100 times slower, something else is wrong.

Re:Cross-platform Rust rewrite of the GNU coreutils by pletnes on Wednesday January 03, 2024 @11:17AM (#38855825)
WSL conflicts with many antivirus/security packages due to e.g DNS filtering or other network restrictions. WSL itself on a non-managed box works not-too-bad, sure.

Cygwin has been very useful to me, but also caused its share of issues with its permissions model, creating large directory trees that couldn’t be deleted.

Re:Cross-platform Rust rewrite of the GNU coreutils by 0x1ceb00da on Wednesday January 03, 2024 @07:14AM (#38853189)
Cross compiling rust programs is much easier, so if you end up on a niche system that doesn't come with coreutils, you can install your own version.
Re:Cross-platform Rust rewrite of the GNU coreutils by ranguna on Wednesday January 03, 2024 @05:43AM (#38852619)
I don't see why I wouldn't switch.
Re:Cross-platform Rust rewrite of the GNU coreutils by wvh on Wednesday January 03, 2024 @09:05AM (#38854162)
The people who originally wrote those utils are slowly disappearing, and with them the knowledge and ancient C hackery skills. It might be a good idea to have a new generation that can take some of that knowledge and interest and carry it forward into the future.
Re:Cross-platform Rust rewrite of the GNU coreutils by tahnyall on Wednesday January 03, 2024 @11:15AM (#38855797)
That'd be great if it were migrating those utilities to modern C using improved modern coding practices but re-writing in some goofy language like rust is dumb.
Re:Cross-platform Rust rewrite of the GNU coreutils by thecodedmessage on Wednesday January 03, 2024 @12:30PM (#38856825)
Writing in Rust when appropriate is improved modern coding practice.
Re:Cross-platform Rust rewrite of the GNU coreutils by ParetoOptimal on Wednesday January 03, 2024 @09:30AM (#38854442)
If it were GPL licensed, I might try switching because I'm more interested in learning Rust than learning more C.
Re:Cross-platform Rust rewrite of the GNU coreutils by nonameiguess on Wednesday January 03, 2024 @10:27AM (#38855162)
Probably makes sense mostly on Mac. Right now, it's common for developers using Macs to install GNU coreutils from Homebrew as a first step to ensure scripts and build systems that were originally written for Linux still work. Macs give greater ease of corporate device management, so Macbooks are a fairly typical compromise for companies that need to give developers a Unix environment but still want to maintain the level of control over endpoints you'd get with Windows.

I'm sure there's desire out there for a pure-Rust close-to-POSIX userspace you can put on top of Linux, but I don't exactly see a whole lot of progress toward that. At minimum, you'd need coreutils, findutils, diffutils, tar, grep, sed, compression libraries, crypto libraries comparable to OpenSSL or GnuTLS, an editor, a POSIX shell, an init system, a service manager, system-wide DNS and DHCP. Going beyond pure POSIX to a more practical server distro would probably include a PGP implementation, a package manager, sudo or equivalent, ssh, iproute2, probably many other things I'm forgetting.

uutils seems to give you findutils and coreutils. ripgrep is not a drop-in replacement for grep. I'm sure there are other "rewrite it in Rust" projects out there I don't know about, but low-level system utilities aren't exactly the gloryland most developers are interested in rewriting and POSIX includes a lot of stuff.

Frankly, I don't think this is a realistic terrain for any single organization. GNU itself took over a decade to get a practical distro and that was only because other developers provided a kernel, package managers, boot loaders, crypto, and what not. And the GNU project is not really a single organization. It encompasses many people from many different employers. Systems software in general wasn't as fragmented. If you were working in the space at all and cared about being cross-platform and collaborating across organizations in the 70s and 80s, you were using C. Everything else was proprietary. If we're ever going to get something comparable but in Rust, it's not going to be from a single Github org like this.

Re:Cross-platform Rust rewrite of the GNU coreutils by asteba on Monday January 15, 2024 @05:37AM (#38999305)
It makes a lot of sense in embedded devices, where the GPLv3 license cannot be used for commercial devices which are using signed images (secure-boot / verified-boot) for security reasons. I expect a lot of adoption there.
Re:Cross-platform Rust rewrite of the GNU coreutils by giancarlostoro on Wednesday January 03, 2024 @10:30AM (#38855201)
> I'm not familiar with the situation on MacOS but how bad is the compatibility issue? Similarly, on Windows what's the situation with WSL or VMs or even Cygwin? Is performance the issue?

If I remember correctly, Mac has any GNU utility from before they all switched to GPL v3. Which was decades ago now.

Re:Cross-platform Rust rewrite of the GNU coreutils by duped on Wednesday January 03, 2024 @10:39AM (#38855305)
The canonical GNU packages are pretty arcane and difficult to build/understand (*), so alternatives with modern tooling/languages should be welcome.

But also, things like BusyBox and ToyBox are very popular alternatives which contain non-standard ports of only a subset of coreutils. A complete and mostly compatible set of coreutils would be more popular than either, but they also have some different constraints that Rust builds make difficult (eg: binary size).

* nb4 "it's just configure/make/install what do you mean" consider if you want coreutils without glibc, cross compiled, or want to bootstrap your environment without any of them. The GNU ecosystem is "easy" to build/use within the GNU ecosystem, which is not unopinionated about what that looks like and how it works.

Re:Cross-platform Rust rewrite of the GNU coreutils by apatheticonion on Wednesday January 03, 2024 @05:46PM (#38860877)
I use them on my Windows machine from PowerShell to make using the Windows command line more tolerable.

You can find the GNU coreutils for Windows on sourceforge but it always feels like spyware when I download something from that site, plus I don't know how old they are or if they are actively maintained.

With this rewrite, I like that I can download them from the repo's releases page. There is an issues page for problems and I can see the project is actively worked on.

Installing them as binaries is simple and, this goes for all Rust/Go projects, I actually know how to compile them (sorry, I know C/C++ is great but between gcc||msvc||clang, missing deps, ./configure ./install and make - I rarely have a pleasant experience compiling something in C/C++).