News for hackers. Stuff that matters.
Re:The Git source code audit, viewed as a Rust programmer by proto_lambda on Tuesday January 24, 2023 @11:33AM (#34505922)
Some of these comparisons are a little of lackluster - sure, the issues may be relatively easy to reproduce if you translate the C code 1:1 to (unsafe) Rust, but you wouldn't even have such code in the first place if you wrote idiomatic Rust:

- Confusing different integer types is much harder not only because the type checker will complain, but also because you don't really come across anything other than usize/isize unless it's a field in a protocol or something.

- Strings in rust are not zero-terminated, so that specific example might not even happen at all.

- Integers, pointers and references are not implicitly cast to booleans, so you can't accidentally mix up comparing a pointer to NULL (which is only ever required for FFI calls) and comparing an integer to 0.

- You have proper types to indicate that a function encountered an error, so all the issues stemming from checking the integer return code the wrong way (e.g. interpreting an error value as a return value) don't exist.

- You simply don't have to invoke potentially dangerous functions like `memcpy` because there are abstractions that take care of this safely and more intuitively.

Re:The Git source code audit, viewed as a Rust programmer by nextaccountic on Tuesday January 24, 2023 @11:44AM (#34506106)
> - Confusing different integer types is much harder not only because the type checker will complain, but also because you don't really come across anything other than usize/isize unless it's a field in a protocol or something.

For me it's the opposite: usize is only used for indexing arrays, isize only used for memory offsets (which is a really niche use case outside of low level code), and most application code should be written with u32, u64, i32 and i64. Why is that? Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic.

So you often need to index something based on an u32 (or something) and then there's a cast. How do avoid it?

Well first off, your own types can implement Index<u32> instead of Index<usize>. But better yet, you can have an Idx type that represents an index, and then implement Index<Idx>. That's seen for example in the petgraph API [0], you can index an graph with either a node index [1] or an edge index [2]

[0] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph....

[1] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph....

[2] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph....

Re:The Git source code audit, viewed as a Rust programmer by saghm on Tuesday January 24, 2023 @12:21PM (#34506730)
> So you often need to index something based on an u32 (or something) and then there's a cast. How do avoid it?

I imagine the nuance here is in the "or something", since that could imply something signed or larger than 32 bits, but for that specific case, I feel like a large amount of code that people write nowadays can be comfortably assumed not to need to run on 16-bit processors. Although `usize `only implements `TryFrom<u32>` and not `From<u32>`, even the standard library documentation for Rust simplifies things by saying that `usize` will be either 4 or 8 bytes (https://doc.rust-lang.org/std/primitive.usize.html), although I understand this isn't a hard contract and could change in the future. I guess it depends a bit on the use case, but I think that most of the time just using `try_from` and panicking with an appropriate error message on error would be fine, and if you want to be extra sure, you can always use something like `const_assert!` (https://docs.rs/static_assertions/1.1.0/static_assertions/ma...) with the `BITS` constant (https://doc.rust-lang.org/std/primitive.usize.html#associate...) to catch it at compile time to be extra certain.

Re:The Git source code audit, viewed as a Rust programmer by nextaccountic on Tuesday January 24, 2023 @12:48PM (#34507175)
Oh, u32 is generally fine exactly because 16 bits is so niche. But you still have a cast between u32 and usize (and .into() isn't that much better). Casts or conversions on each collection access makes code hard to read

So if your application can work with u32 everywhere (on either 32 bits or 64 bits platforms), you should use u32 and impl Index<u32> on your collection, to avoid the cast.

It's only if you need the less common use case of u32 on 32 bits platforms, u64 in 64 bits platforms that you should bother with usize

Re:The Git source code audit, viewed as a Rust programmer by phkahler on Tuesday January 24, 2023 @01:39PM (#34507973)
>> So if your application can work with u32 everywhere (on either 32 bits or 64 bits platforms), you should use u32 and impl Index<u32> on your collection, to avoid the cast.

This seems strange. Why does Rust not automatically use whatever size index you hand it?

Re:The Git source code audit, viewed as a Rust programmer by db48x on Tuesday January 24, 2023 @02:12PM (#34508546)
Ultimately it is because the Rust developers decided that there would be no implicit conversions. In a lot of cases implicit conversions are convenient, but many people feel that the complexity just isn’t worth it.
Re:The Git source code audit, viewed as a Rust programmer by Sharlin on Tuesday January 24, 2023 @03:30PM (#34509836)
It’s not out of the question that additional `Index` impls for other integer types could be added to the standard library at some point; the issue is discussed often enough. But it’s not trivial to do because some currently valid code could become ambiguous if impls are added, so some sort of a transition would be required.
Re:The Git source code audit, viewed as a Rust programmer by proto_lambda on Tuesday January 24, 2023 @11:58AM (#34506341)
> Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic.

Fair point, but it really only matters if the overflow is handled gracefully. If it's handled as an unrecoverable error condition, it often 1. doesn't happen during normal operation and 2. can be invoked by a determined attacker, regardless of the exact size. Sure, the exact characteristics might be different, but in the end it doesn't make much of a difference.

Petgraph has a great API, and in general using newtypes over integers for collection indices is a great way to avoid another logic bug: mixing up indices for different collections.

Re:The Git source code audit, viewed as a Rust programmer by nextaccountic on Tuesday January 24, 2023 @12:49PM (#34507182)
Defining a newtype avoids confusing indexes of, say, a graph and a Vec, but doesn't avoid confusing indexes of two graphs

Rust has a design pattern called lifetime branding (also called generativity), which uses phony lifetimes to prevent, at compile time, confusing indexes of two separate collections of the same type. This can also enable disabling out of bounds checks without triggering UB (because, with branding, we can be sure that the index is on bounds at the moment of creating it; essentially we move bounds check from the indexing time to the index creation time)

Here's an earlier mention of that [0] (7 years ago), and here's a crate from 3 years ago, indexing [1] but I'm not sure about recent developments on that.

Now, petgraph doesn't use branding for its index types, so if you have two graphs you can confuse their indexes. On the other hand, petgraph was specifically designed so that you can reuse the node and edge numbering across many graphs (so that if you have a subgraph for example, the nodes and edges share the same ids), in this situation it's kind of hard to use branding

There's another pattern for not confusing index types which is to make different index types for each different collection and make the collection work only with that type; this is done eg. in typed-indexed-collections [2] - but it doesn't use branding so two collections with same index type have interchangeable indexes

Anyway right now this stuff is mostly folklore but I wish it were more used.

[0] https://www.reddit.com/r/rust/comments/3oo0oe/sound_unchecke...

[1] https://github.com/bluss/indexing https://docs.rs/indexing/0.4.1/indexing/

https://crates.io/crates/typed-index-collections https://www.reddit.com/r/rust/comments/hr6xcu/announcing_typ...

Re:The Git source code audit, viewed as a Rust programmer by marcosdumay on Tuesday January 24, 2023 @12:18PM (#34506677)
I'm not sure it's the best policy, but I tend to represent every counter, of anything as usize. It doesn't matter if it's a memory map.

I imagine that applications with less addressing space also have less of every other resource, and can count less of everything.

Re:The Git source code audit, viewed as a Rust programmer by nextaccountic on Tuesday January 24, 2023 @12:29PM (#34506879)
> I imagine that applications with less addressing space also have less of every other resource, and can count less of everything.

That's a fair point, and I've seen here on HN a design approach that everything should have a limit (because programs often explode in unexpected ways, such as degraded performance, when operating outside the programmer's expectations)

But, usize doesn't necessarily capture the right limit, so.. I'm not sure

Re:The Git source code audit, viewed as a Rust programmer by marcosdumay on Wednesday January 25, 2023 @03:30PM (#34523762)
Well, yes. I favor your program blowing up on integer overflow too, so it's very predictable how it behaves. And if it's not overflow that is the concern here, well, it's not solved by going with another type either.

Almost all the time, usize has a convenient limit that is much larger than needed. On the few exceptions, yes, you have to design your types and deal with type conversions.

Re:The Git source code audit, viewed as a Rust programmer by yakubin on Tuesday January 24, 2023 @01:25PM (#34507735)
It's sometimes good to use a smaller size. Many string/dynamic array libraries e.g. will use 32-bit types for size and capacity even on 64-bit platforms to conserve space, and by that virtue also be more cache-efficient. I think the Zig compiler also explicitly uses 32-bit types in a lot of places, even though it's targeted mostly for 64-bit platforms.
Re:The Git source code audit, viewed as a Rust programmer by stouset on Tuesday January 24, 2023 @12:44PM (#34507104)
Also what’s fun is that when you have usize and u64, which are both the same size on 64-bit systems, you might instinctively cast between them. But this could truncate the u64 on 32-bit systems, and it could truncate the usize on future 128-bit systems!

So neither type implements From/Into the other, the best you get are TryFrom/TryInto where you either have to handle an error or discard it after “proving” overflow can’t happen.

This is unambiguously the correct behavior, but it is sort of frustrating at times.

Re:The Git source code audit, viewed as a Rust programmer by AnimalMuppet on Tuesday January 24, 2023 @01:05PM (#34507412)
> For me it's the opposite: usize is only used for indexing arrays, isize only used for memory offsets (which is a really niche use case outside of low level code), and most application code should be written with u32, u64, i32 and i64. Why is that? Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic.

While that may make it platform agnostic, doesn't that make it non-time-agnostic? What I mean is, I lived through the 16-to-32-bit transition. If Rust had been around then, and you wrote code that used u16, and you tried to use it now, you would probably bump against that 16-bit limit. The platforms grew with time, but the code didn't.

We're starting into the 32-to-64-bit transition (we've been slowly easing into it for years, in fact). Don't lock yourself into 32-bit code (unless the algorithm requires it). You're locking yourself out of the future.

Re:The Git source code audit, viewed as a Rust programmer by nextaccountic on Tuesday January 24, 2023 @01:24PM (#34507713)
64 bits is huge! We won't be needing more than 64 bits to address memory anytime soon. In fact, current processors use only 48 bits, leaving the rest unused / up for grabs with flags.

And note that the usize that is the index of Vec (and arrays, etc) is not an address but just a counter; it can count up to more than 10^19 elements. You probably won't be alive to see Vecs grow that big, even a Vec<u8>.

The most interesting platform with 128 bits pointers is CHERI, but its address space fits on 64 bits; the other 64 bits is used for a tag, to validate pointers (and prevent certain kinds of buggy memory access that can lead to corruption). If and when Rust support CHERI it should probably do so with usize = 64 bits, because that's how much you need to count to address a Vec<u8> over the whole address space.

Re:The Git source code audit, viewed as a Rust programmer by phkahler on Tuesday January 24, 2023 @01:48PM (#34508127)
I've often thought a 48bit architecture might be ideal. 256TB of address space. Use 48bit posits instead of double precision floating point for similar precision. A 48bit pixel could be 16bit rgb or 12bit rgba. I've also tried to re-encode the RISC-V instruction set and I think 24-bit opcodes with immediate data following them would be better than what they've got. Here's to 48bit CPUs! Hahaha.
Re:The Git source code audit, viewed as a Rust programmer by AnimalMuppet on Tuesday January 24, 2023 @05:19PM (#34511388)
64 bits is huge... for addressing memory. That's what you were already using usize for, though. What about the other places, where you're using u32? Is 64 bits huge for them? All of them? Will it still be huge for all of them in 10 years? 20?
Re:The Git source code audit, viewed as a Rust programmer by alexvoda on Tuesday January 24, 2023 @12:05PM (#34506467)
Indeed, over the decades, the field moved from static weak typing (with a few languages being strong) to dynamic weak typing to strong typing with type inference.

C has not benefited from any of this progress.

Re:The Git source code audit, viewed as a Rust programmer by kevin_thibedeau on Tuesday January 24, 2023 @02:51PM (#34509209)
Modern C compilers have these features. They just need to be enabled.
Re:The Git source code audit, viewed as a Rust programmer by alexvoda on Sunday January 29, 2023 @07:40AM (#34567555)
Maybe. But old code does not benefit from them. And new code either has to deal with old code or is often written in a style that does not use them. They are opt-in.

More modern languages have certain features enabled by default. They are either mandatory or opt-out.