News for hackers. Stuff that matters.
Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by Xeoncross on Thursday January 25, 2024 @06:32PM (#39136940)
It is great when there is a standard that works well for everyone.

It is great when there is competition to that bloated monopoly everyone currently uses.

I don't understand my own cognitive dissonance on this topic.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by saintfire on Friday January 26, 2024 @01:10AM (#39139408)
I think the competition can drive the standard to improve.

People can see improvements in alternatives that still align with their principals and adopt them.

So in that regard, I don't think they are disjunct ideas.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by Klonoar on Thursday January 25, 2024 @07:15PM (#39137307)
I want to make a sign on the wall that denotes $x days since someone complained about Tokio + Reqwest being bloated and/or making their own design decisions.

I also want to make a campaign to direct people to ureq first so we stop hearing about this every other month. Reqwest is very good and worth just using IMO, but if you really care about this, ureq is pretty battle tested and a fine option for "just give me a fucking HTTP client".

(A selfish third point but ureq needs a happy eyeballs implementation, but that's neither here nor there I guess)

Edit: I did read the article and yes, I am commenting something slightly tangential

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by pcwalton on Thursday January 25, 2024 @07:20PM (#39137344)
Yeah, it seems to me like the author just wants a simple HTTP client for Rust that prioritizes simplicity over raw performance and is unhappy with the fact that reqwest isn't that. Though as developers we often wish it were otherwise, given a choice between implementation simplicity and performance most people will choose the latter (as long as correctness isn't sacrificed). So that's why reqwest is more popular and is the first choice that appears when searching for [rust http client] on Google.

Thankfully, ureq exists for those who prefer implementation simplicity, and that's what the author should probably choose.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by Scramblejams on Thursday January 25, 2024 @08:27PM (#39137860)
I read it slightly differently.

> the author just wants a simple HTTP client for Rust that prioritizes simplicity over raw performance

I saw it as:

> the author just wants a safe HTTP client for Rust that prioritizes safety over raw performance

That resonates with me as well.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by mrkline on Thursday January 25, 2024 @08:35PM (#39137916)
That's my main goal - and Rust seems like the language where we can have our cake and eat it too! The standard library has some amazingly performant data structures and algorithms.

Talking to one of the hyper developers, a lot of this machinery was made before it arrived in the stdlib. My response was, "neat, so we can delete the custom stuff now, right? :P" but I suppose any change carries some risk with it.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by amluto on Thursday January 25, 2024 @07:07PM (#39137229)
> or even just skipping blank lines, hyper does things its own way.

Ugh. IMO a bit benefit of Rust is that you can’t do wild-west-YOLO buffer twiddling without “unsafe,” so people will write better code.

But it really looks like httparse missed the memo.

https://github.com/seanmonstar/httparse/blob/v1.8.0/src/iter...

iter::Bytes looks like an awkward wrapper around slices with all the safety removed. So you can port nasty C-style code right over.

Seriously, it should not be hard to efficiently strip a prefix off a u8 slice in safe Rust. For example, split_first.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by LegionMammal978 on Thursday January 25, 2024 @07:36PM (#39137474)
These days, stripping fixed-length prefixes and suffixes off of slices is pretty easy with slice patterns. But that code was originally written around the time of Rust 1.0.0, when the language and standard library had far fewer features to help with this. For instance, it uses slice::from_raw_parts() rather than slice.get_unchecked(pos..), since the latter wasn't stable until Rust 1.15.0. Similarly, split_first() wasn't stable until Rust 1.5.0, and slice patterns weren't stable until Rust 1.26.0.
Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by scns on Thursday January 25, 2024 @08:39PM (#39137945)
> Seriously, it should not be hard to efficiently strip a prefix off a u8 slice in safe Rust. For example, split_first.

That is a good idea for a contribution, no?

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by andrewaylett on Friday January 26, 2024 @05:21AM (#39140958)
Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by andrewaylett on Friday January 26, 2024 @02:11PM (#39146702)
Follow up -- the benchmarks on that PR look poor, but the benchmarks don't actually quite measure what they're expected to measure -- they're not black-boxing inputs, so the compiler has an opportunity to constant-fold in some cases.

I raised a PR to fix the benchmarks: https://github.com/seanmonstar/httparse/pull/151

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by nindalf on Thursday January 25, 2024 @07:27PM (#39137407)
There’s a small message contained in that unnecessarily long post - that maybe a popular library used for HTTP in Rust could use less unsafe.

There was no need for this half mocking, half condescending tone. If the author wanted an explanation for a technical decision, they could open an issue and have a conversation like an adult. Instead we’re left with their speculation that leads nowhere. They cry about the existence of some unsafe code, but don’t actually put in the effort to figure out if it can lead to a real problem like unsoundness. And somehow the title implies that they’re saying something profound about the entire rust ecosystem when they just looked at one library. It’s just innuendo, as far as I could tell.

Here’s a post by the maintainer of hyper about what they accomplished in 2023 and what they hope to accomplish in 2024 - https://seanmonstar.com/blog/2023-in-review/

If this work interests you, or you depend on hyper in production like many companies do, then consider sponsoring them! Or maybe you could give back to the commons by submitting PRs that fix issues you’ve found. Or even a good bug report would be appreciated.

But not this kind of article. This article helps no one and does nothing constructive. We all benefit from the work that open source maintainers put in. They have it hard enough without having to read low effort posts trashing their work. Be better.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by lmm on Thursday January 25, 2024 @07:35PM (#39137470)
> If the author wanted an explanation for a technical decision, they could open an issue and have a conversation like an adult.

Opening an issue comes across as a whole lot more aggressive to me. That implies that they owe you an explanation or you want them to change their code.

> They cry about the existence of some unsafe code, but don’t actually put in the effort to figure out if it can lead to a real problem like unsoundness.

unsafe might as well be unsound. The key benefit of Rust is supposed to be better memory safety than C; if everything is using unsafe, why bother?

> Here’s a post by the maintainer of hyper about what they accomplished in 2023 and what they hope to accomplish in 2024 - https://seanmonstar.com/blog/2023-in-review/

Ok, and that doesn't mention "unsafe" once - no justification for why it's used, much less a plan to reduce or eliminate it.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by nindalf on Thursday January 25, 2024 @07:44PM (#39137550)
> Opening an issue comes across as a whole lot more aggressive to me. That implies…

It implies nothing. You can politely open an issue and ask why HeaderMap is used instead of HashMap. As long as you’re polite, it’ll be fine.

> unsafe might as well be unsound.

Oh boy, hope you don’t open Vec in the standard library. It can’t be implemented without unsafe and it’s used by almost all Rust programs. I guess every program is unsound, according to you?

> Ok, and that doesn't mention "unsafe" once

Nor does it need to. Just because OP decided this is a problem doesn’t mean it actually is a problem. Find unsoundness and then you have something worth discussing.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by lmm on Thursday January 25, 2024 @10:35PM (#39138635)
> It implies nothing. You can politely open an issue and ask why HeaderMap is used instead of HashMap. As long as you’re polite, it’ll be fine.

I have direct personal experience that opening an issue to politely ask this kind of design question is not always fine.

> Oh boy, hope you don’t open Vec in the standard library. It can’t be implemented without unsafe and it’s used by almost all Rust programs. I guess every program is unsound, according to you?

Based on past experience with LLVM undefined behaviour, yes, I think it's highly likely that every existing nontrivial Rust program is unsound in current Rust. But "depends on a standard library class that is currently implemented unsafely" will be much easier to fix in the future than "uses unsafe itself".

> Nor does it need to. Just because OP decided this is a problem doesn’t mean it actually is a problem. Find unsoundness and then you have something worth discussing.

Again, why use Rust at all if that's how you feel?

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by nindalf on Friday January 26, 2024 @02:02AM (#39139646)
You have no idea what you’re talking about. You’re so confidently incorrect about Rust that there’s no point even trying to educate you. Good day.
Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by dang on Monday January 29, 2024 @01:32AM (#39173389)
Please don't post like this. We have to ban such accounts and I don't want to ban you.

See https://news.ycombinator.com/item?id=39173369 for more.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by timeon on Friday January 26, 2024 @06:15AM (#39141348)
> The key benefit of Rust is supposed to be better memory safety than C; if everything is using unsafe, why bother?

Is Hyper all contained inside unsafe block? As far as I know it is not. So even if there is some unsafe code it is not same as writing it in C.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by lmm on Friday January 26, 2024 @08:39PM (#39151460)
Apparently even quite basic things like splitting a string are done in unsafe code. So it sounds like while it may not be 100% unsafe, most of it is.
Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by mrkline on Thursday January 25, 2024 @07:35PM (#39137468)
The article got me in touch with one of the hyper devs, and we had a very friendly conversation that taught me a lot.

If questioning some design decisions and calling a marketing blurb a little disingenuous is mocking and condescending, they didn't seem to think so.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by nindalf on Thursday January 25, 2024 @07:39PM (#39137508)
They told you they enjoyed your article? Or did you just infer that because they engaged politely?

All of my criticisms about your article still stand. It contributed nothing worthwhile, but I’m happy it got you your 15 minutes on HN.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by Scramblejams on Thursday January 25, 2024 @08:21PM (#39137812)
> It contributed nothing worthwhile, but I’m happy it got you your 15 minutes on HN.

That is a very rude dismissal, followed by a bad faith implication of the author's motivations. You can do better.

I got something worthwhile out of it. I'm learning Rust and, like the author of the blog post, tend to value safety (simplicity hopefully being a contributor to that) over that last bit of performance. I was actually looking over the source of some of the libraries I'm considering using, am concerned about some things I found, didn't know about ureq, now I do, and I think it's going to fit my needs.

Thanks, mrkline!

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by worik on Friday January 26, 2024 @02:42AM (#39139842)
I got a lot out of it too.

I have been using Rust, casually, for about eight years.

Sometimes in Rust it is hard to find simple things that solve simple problems.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by nindalf on Friday January 26, 2024 @03:10AM (#39139992)
The author put the absolute bare minimum of effort into this article. He condescendingly points out that hyper uses patterns and code that could be pulled from stdlib.

If he had had a conversation with the maintainer he could have found out and told us. Turns out he asked after writing this. Hyper was written before those pieces were added to the stdlib. Changing it would simplify the codebase, but there’s always a risk of breaking a mature, battle tested codebase. See how this new information completely changes the tone of the article?

Anyway, the crate is being actively maintained. It’s possible the author will look into making these improvements at some point. Until then the entire community can continue to benefit from a battle tested, performant, secure http lib.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by mrkline on Thursday January 25, 2024 @07:45PM (#39137555)
Turns out we regularly play flight sims together on weekends, but didn't know about each others' software misadventures - small world, isn't it?

If the reason I blogged about my coding hobby was to "get my 15 minutes on HN", I would be a deeply unhappy person.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by nindalf on Thursday January 25, 2024 @08:00PM (#39137664)
I’m happy for you. Do better next time. Good day.
Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by dang on Monday January 29, 2024 @01:29AM (#39173369)
You broke the site guidelines badly and repeatedly in this thread, for example in your aggressive comments to the article author, and in https://news.ycombinator.com/item?id=39139646 which was really totally unacceptable.

This is not cool.

You're a good contributor, and I appreciate that you don't do this all the time, but we've also had to warn you multiple times about this kind of thing in the past. If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by marcinzm on Thursday January 25, 2024 @07:12PM (#39137273)
> Simpler is better: Complex problems require complex solutions, but we should strive for simplicity in our software. Code is a liability—less code is less that could go wrong, and less to debug when things do go wrong!

This seems to confuse complexity with features. If a library has 100k lines of code but the parts you use are only 1k lines of code then why is that worse than a library with 1k lines of code?

> I want a single HTTPS connection. I don’t need persistent sessions with connection pools and cookies. I don’t need an async runtime.2 I need a glass of scotch, a socket, and a few syscalls. In the same vein,

That's true right until the moment you do need them and then you need to rewrite a lot of code. Especially great fun when the thing you're connecting to assumes everyone has this feature and you spend hours or days debugging things.

Modular, popular and well structured libraries with all the features one might reasonably need are my preference. Minimalism in lines of code is as much a trap as a minimalism in benchmark performance.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by mrkline on Thursday January 25, 2024 @07:26PM (#39137399)
> If a library has 100k lines of code but the parts you use are only 1k lines of code then why is that worse than a library with 1k lines of code?

There's nothing wrong with not using the whole feature set of some library. But if library A does the same things as library B with a third of the code, isn't that better? (All other things - e.g. perf - being equal)

> That's true right until the moment you do need them and then you need to rewrite a lot of code.

There's plenty of applications that will only ever need a handful of connections. Probably most applications.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by lmm on Thursday January 25, 2024 @06:40PM (#39137015)
> Do huge swaths of Rust users value vanishingly small performance gains over memory safety, in a language that prides itself on being able to provide speed and safety?

I mean, yeah, probably? It was meant to appeal to people who were still writing things in C/C++, who are by definition people with those values. For general-purpose use the language was always "OCaml but with vanishingly small performance gains".

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by api on Thursday January 25, 2024 @06:32PM (#39136944)
Rust indeed does have an embarrassing excess of diversity problem, made worse by the fact that async isn't in the standard library (outside bare minimum stuff like Future) and async libraries can't really be async runtime neutral. (This is going to improve with async traits.)

It's probably an inevitable result of Rust having a thinner standard library than Go, Java, C#, or other heavier languages. It's supposed to be systems language after all.

Another reason though is that Rust is a rich language with a very powerful type system, and rich powerful languages invite programmers to show off. One of the greatest things about Go is how boring it is. It doesn't give you a lot of room to show off by writing clever code, so instead you have to show off by making a great application. But some of this is unavoidable if you want what Rust delivers: a near-zero-overhead systems language with hard safety and automatic memory management without GC. That's dumping a heavy load on the type system, and Rust does deliver pretty well.

I try to exercise discipline when writing Rust and not show off by being more clever than I need to be. I do write some hand-whittled performance code here and there in high-performance applications, but I try really hard to avoid using unsafe. I've found that you usually can.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by worik on Thursday January 25, 2024 @06:56PM (#39137143)
> by the fact that async isn't in the standard library

I wish async/await were not in the language

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by sliken on Thursday January 25, 2024 @07:16PM (#39137317)
So just ignore the async use cases? Or force tokio or similar to support it?

My biggest pain point so far is channels, I like them in go because they are straight forward to use, allow concurrency, aren't a minefield of problems, and are easy to reason with. I wanted to scan N directories (in parallel) -> encrypt and checksum in (parallel) -> queue files for upload. With go channels it's straight forward.

I was looking for MPMC (like go channels) to allow multiple producers to enqueue and multiple consumers to dequeue. Rust doesn't do it and tokio doesn't do it. I could use crossbeam[0], flume[1], or async[2]. Not sure what limitations those have and what if any limitations in compatibilities with other crates I need will be.

[0] https://docs.rs/crossbeam/latest/crossbeam/channel/index.htm...

[1] https://docs.rs/flume/latest/flume/

[2] https://docs.rs/async-channel/latest/async_channel/

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by LegionMammal978 on Thursday January 25, 2024 @07:50PM (#39137586)
FWIW, either async-channel's types or flume's async APIs should work for your use case. Both are completely agnostic to the async runtime, since the futures are triggered by internal events (recv() waits for send() and vice versa), rather than external events like I/O which are generally coordinated through the runtime.
Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by worik on Friday January 26, 2024 @02:45AM (#39139859)
> So just ignore the async use cases? Or force tokio or similar to support it?

Hell no!

Do it differently

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by aw1621107 on Friday January 26, 2024 @04:36AM (#39140566)
Differently how?

And perhaps more importantly, what tradeoff(s) would those differences entail?

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by Ygg2 on Friday January 26, 2024 @10:20AM (#39143622)
Green threads/Virtual threads are one solution, but then you need to ship a runtime for them, then you pay for what you don't use.

Or don't standardize on them and have a split ecosystem of half libs supporting use `green-thread.rs` and the other half supporting `rust-gthreads`.

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by api on Friday January 26, 2024 @04:36PM (#39148820)
I wonder if you could have hidden green threads from the application entirely. Have a "full" standard library where std::thread is M:N green threads and I/O is implemented behind the scenes that way, and a "thin" version where std::thread is OS threads and I/O is just passed through to the OS. (Then of course no_std for embedded.)

Of course the problem then would be that Rust makes it trivially easy to call C code. As soon as someone in a green thread runtime called libc I/O functions directly they'd be in for a weird surprise when the call froze all threads on their worker.

Of course the "full" M:N runtime could be instrumented to print a warning when this occurs unless explicitly told not to, and std::thread in the "full" stdlib could allow explicit creation of full OS threads outside the runtime.

There is an attempt at doing this out there:

https://github.com/Xudong-Huang/may

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by aw1621107 on Friday January 26, 2024 @05:45PM (#39149846)
> Have a "full" standard library where std::thread is M:N green threads and I/O is implemented behind the scenes that way, and a "thin" version where std::thread is OS threads and I/O is just passed through to the OS. (Then of course no_std for embedded.)

That sounds vaguely like what Rust used to have with its I/O functions before its green thread implementation was removed [0]? Not sure it's quite the same thing, though:

> In today's [2014] Rust, there is a single I/O API -- std::io -- that provides blocking operations only and works with both threading models. Rust is somewhat unusual in allowing programs to mix native and green threading, and furthermore allowing some degree of interoperation between the two.

> [snip]

> In this setup, libstd works directly against the runtime interface. When invoking an I/O or scheduling operation, it first finds the current Task, and then extracts the Runtime trait object to actually perform the operation.

> On native tasks, blocking operations simply block. On green tasks, blocking operations are routed through the green scheduler and/or underlying event loop and nonblocking I/O.

[0]: https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by steveklabnik on Friday January 26, 2024 @06:21PM (#39150248)
May has soundness issues around TLS. You can cause undefined behavior in safe code. They do not plan on changing that (and I'm not sure it's even possible) but there's no way that it will become super popular due to this issue.

(and yes, as my sibling commentor mentions, Rust tried this before. Doesn't mean nobody can do it, but I suspect that it is not possible.)

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by aw1621107 on Friday January 26, 2024 @05:49PM (#39149885)
> Green threads/Virtual threads are one solution, but then you need to ship a runtime for them, then you pay for what you don't use.

Indeed, and IIRC that's one of the reasons Rust removed its green thread implementation in the first place.

> Or don't standardize on them and have a split ecosystem of half libs supporting use `green-thread.rs` and the other half supporting `rust-gthreads`.

Don't forget the third lib for abstracting over the two!

Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by Ygg2 on Thursday January 25, 2024 @07:13PM (#39137279)
However other people wished it was. So here we are.
Re:Unsafe at Any Speed: Tradeoffs and Values in the Rust Eco... by 38 on Thursday January 25, 2024 @06:39PM (#39137007)
> It doesn't give you a lot of room to show off by writing clever code, so instead you have to show off by making a great application.

I agree with your comment as a whole, but I have also seen plenty of dumb Go code. hey, lets toss out all compile time type checking:

https://johnstarich.com/go/pipe/pkg/github.com/johnstarich/g...