But many interesting data structures are hard to write in a memory-efficient manner without resorting to non-contiguous nodes. Even a hashtable often will use linked lists within each bucket for collision resolution. You can argue that an open-addressing scheme is more cache-friendly, but it also has downsides, i.e. performance degrades faster as the load factor gets higher.
Many other interesting data structures, especially some lock-free structures, are simply impractical to implement as single contiguous arrays.
Of course, any node-based structure can make use of a memory pool that allocates blocks from one or more larger contiguous buffers, but there will still be pointers interleaved throughout the structure.
All in all, it remains true that Rust doesn't really provide any safety above C++ in regard to writing these kinds of node-based data structures, and saying "don't ever write node-based data structures" is just pointless. Yes, Rust has some downsides and tradeoffs. Is it so bad to just say that out loud?
Yup! Rust does not fix all your problems, sadly. Maybe some of them. :-)
If you're working with graph-like structures in Rust, then you have two choices:
1. You can work with pointers using 'unsafe' blocks, and then wrap everything inside a nice, safe API. This is a good tradeoff if your data structure is only a tiny portion of your code, or if it's reusable. (Most of the data structures in Rust's 'std' crate are written like this.)
2. You can work with indices instead of pointers. This is the approach taken by petgraph (https://docs.rs/petgraph/0.4.3/petgraph/), which is currently one of the best graph libraries for Rust.
Not all Rust code needs to be safe. It's OK to write "unsafe" code and put it behind a "safe" API. If I can eliminate 98% of the opportunities for pointer bugs, I'm happy to eyeball the remaining 2% manually.
Debatable. It certainly does for the consumers of these structures, and data structures are consumed a lot more often than they are rewritten.
> Yes, Rust has some downsides and tradeoffs. Is it so bad to just say that out loud?
Of course not. But that doesn't mean you shouldn't expect some pushback if you're incorrectly describing those tradeoffs.
Is it really so bad to say "at the end of the day, the difficulty of complex graphs in safe Rust is symptomatic not of a flaw in Rust but of how hard it is to build these things without memory errors even in C or C++" out loud?
Also if you're trying to implement lockfree data structures then safe/unsafe pointer access are going to be the least of your worries :).
Rust will make it tricky to implement these, but then you can use the datastructure safely as much as you want. It's a "write once use everywhere" thing.
Or it's really hard to find the good one amongst a hundred bad ones (to paraphrase ESR, when he tried Rust in all seriousness).
"Write once use everywhere" is wishful thinking, it's important to make adaptions (to paraphrase Knuth).
I'm not sure I comprehend this, since I don't often encounter code where people use advanced data structures to do simple things. For most collections, you're typically talking about an array of some kind, since you just want to iterate through many objects doing the same thing. However, there is a reason for more complex data structures to exist. KDTrees, for example, are great for performing nearest-neighbour search, and I can't see any situation where I'd want to use a straight array if I knew I was doing nearest-neighbour searches.
In the most broad, general case I agree: array-of-struct or struct-of-arrays is probably the way to go. You can even see this in the data science world where data frames (effectively struct-of-arrays) are the tool of the trade. However, there are algorithms where if you want an advanced data structure, you really want it (e.g. KDTree). Is it really so common in systems programming to see people trying to incorporate all sorts of magic from The Art of Computer Programming into their programs? I can't say I've ever seen it myself (obvious disclaimer that while I'm a programmer by trade I don't read too many large Rust / C++ projects).
SoA + indices, because locality of the search is critical and you want explicit control over the layout.
I wasn't putting it out there as a dig against C/C++ but more a general comment on how rarely it's understood/appreciated in native code.
Yes! This is a point that I try to hammer home that's missed by many people who write C/C++ on a daily basis. Everyone wants to use the fanciest data structures when most of the times arrays will be faster and simpler to use.