News for hackers. Stuff that matters.
Re:Rust: Not So Great For Codec Implementing by Animats on Tuesday August 01, 2017 @01:00PM (#14902852)
The only legitimate complaint there is that it's tough in Rust to get two arbitrary mutable slices into the same array. Rust wants to be sure they're disjoint, to prevent aliasing. For some matrix manipulation, this is inconvenient.

It would be easier if Rust had real multidimensional arrays. If the compiler knows about multidimensional arrays, some additional optimizations are possible. For example, if you want to borrow two different rows, that's fine as long as the row subscript is different.

Re:Rust: Not So Great For Codec Implementing by pcwalton on Tuesday August 01, 2017 @01:17PM (#14903028)
As the article notes in an edit, there's a method for this: split_at_mut(). It's very useful, and I reach for it quite a bit in low-level array code.

It might be nice to have some sort of pattern matching syntax for it, I suppose:

    let [ref mut a..16, ref mut b..] = *c;
Meh. Sure is ugly. I'm not sure adding syntax would be worth it.
Re:Rust: Not So Great For Codec Implementing by comex on Tuesday August 01, 2017 @06:07PM (#14905843)
I feel like split_at_mut is a bit of a red herring - sure, it's exactly what the author was looking for and unable to find, but it's masking a bigger problem/opportunity here.

Why does the author have to worry about aliasing in the first place? If they're dealing with POD types, there's no memory safety issue implicated by allowing overlapping slices (except possibly when accessing from multiple threads). No-alias guarantees can improve performance in many cases (though rustc currently isn't doing the best job conveying them to LLVM), but not always, and especially in something like a video codec, the author may prefer to manually optimize memory access patterns.

In my own code, I worked around this by using &[Cell<u8>] as an aliasable mutable buffer type. I had a function to cast to it from &mut [u8], as well as functions to read and write integers of various sizes by directly casting the pointer (although this may not be necessary, as LLVM can probably optimize the naive approach of calling .get()/.set() on each Cell individually). Simple enough, and I'm pretty sure it's safe, but I've never seen anyone else use this approach, and I don't know of any crates.io crates that implement it. Maybe I should write one...

Re:Rust: Not So Great For Codec Implementing by dbaupp on Tuesday August 01, 2017 @06:41PM (#14906149)
It is safe, and was even modeled explicitly as part of https://www.ralfj.de/blog/2017/07/08/rustbelt.html .

There's the external https://crates.io/crates/alias and even an approved RFC for adopting it into std: https://github.com/rust-lang/rust/issues/43038 .

Re:Rust: Not So Great For Codec Implementing by comex on Tuesday August 01, 2017 @06:54PM (#14906245)
Doh, way ahead of me then. Though it seems there are more steps needed, e.g. adding support to the ‘bytes’ crate.
Re:Rust: Not So Great For Codec Implementing by phaylon on Tuesday August 01, 2017 @03:24PM (#14904397)
Well, if we're having some bike-shedding fun:

    let (a, b) = c[..16..];
Re:Rust: Not So Great For Codec Implementing by sitkack on Wednesday August 02, 2017 @01:04AM (#14908064)
.split_at_mut is a method on a slice ... one is free to implement their own abstractions that ultimately terminate in an unsafe { block } that enable multiple mutable borrows.

   fn split_at_mut(&mut self, mid: usize) -> (&mut str, &mut str) {
		// is_char_boundary checks that the index is in [0, .len()]
		if self.is_char_boundary(mid) {
			let len = self.len();
			let ptr = self.as_ptr() as *mut u8;
			unsafe {
				(from_raw_parts_mut(ptr, mid),
				 from_raw_parts_mut(ptr.offset(mid as isize), len - mid))
			}
		} else {
			slice_error_fail(self, 0, mid)
		}
	}
this is literally the src to .split_at_mut Make your own version. Rusts power isn't what it enables, it is what it removes, undefined behavior, false sharing, etc. Frankly, I am tired of C/C++ programmers complain about Rust. This is a non-starter. I am happy to convert the F#, Haskell, OCaml, Scala and Clojure devs to Rust. Systems programming isn't magic.
Re:Rust: Not So Great For Codec Implementing by wbl on Wednesday August 02, 2017 @11:50AM (#14911495)
What does Rust have Ocaml doesn't? If Ocaml solves the problem why Rust? Rust was made to solve the problem Ada solves.
Re:Rust: Not So Great For Codec Implementing by kbenson on Wednesday August 02, 2017 @12:48PM (#14912201)
> What does Rust have Ocaml doesn't?

Safe memory management without a garbage collector.

> If Ocaml solves the problem why Rust?

Ocaml doesn't solve every problem any more than Rust does. Rust may solve a subset of the problems people use the listed languages for better than those languages, so it might be beneficial for people to use it in those cases.

> Rust was made to solve the problem Ada solves.

Not all solutions have the same efficiency, or are as easy to understand. Rust aims to enforce safety at compile time to eliminate runtime safety checks where possible. Ada performs many safety checks at runtime, which introduces a performance cost. This does allow Ada to provide some additional safety checks though (such as limiting a type/subtype to a range of values).

Re:Rust: Not So Great For Codec Implementing by sitkack on Friday August 04, 2017 @01:59AM (#14926643)
It has [ ] nothing where a garbage collector would be. ML is beautiful language. I am not trying to get anyone to stop speaking their language of choice, but I am trying to get people who need to do systems-y things to use Rust. There are more similarities between the Rust programmer and the OCaml programmer than there are differences.
Re:Rust: Not So Great For Codec Implementing by myusernameisok on Tuesday August 01, 2017 @01:12PM (#14902978)
> The only legitimate complaint there is that it's tough in Rust to get two arbitrary mutable slices into the same array. Rust wants to be sure they're disjoint, to prevent aliasing. For some matrix manipulation, this is inconvenient.

I've only looked into Rust briefly, but I thought this was the point of unsafe blocks? You can roll your own multi-dimensional array data structure using unsafe blocks if you're certain what you're doing is actually safe.

Re:Rust: Not So Great For Codec Implementing by dbaupp on Tuesday August 01, 2017 @01:24PM (#14903106)
Yes, unsafe blocks one way in which the Rust doesn't privilege built-ins and the standard library (much) over external code: people can write high performance a abstractions without them needing to be baked into the language. This is exactly what has occurred with multidimensional arrays with external libraries like ndarray.
Re:Rust: Not So Great For Codec Implementing by jeeyoungk on Tuesday August 01, 2017 @01:21PM (#14903066)
Unsafe allows you to dereference raw pointers and foreign-function calls, but you cannot go around the borrow checker, which is what's causing annoyance.

That said, you can create raw and dereference raw pointers, which lets you to do the above.

Re:Rust: Not So Great For Codec Implementing by vvanders on Tuesday August 01, 2017 @01:24PM (#14903103)
If you take a ref into a pointer that lets you work around the borrow checker(and I've bitten myself just about every time I've done it now that I'm used to the borrow checker).
Re:Rust: Not So Great For Codec Implementing by wuch on Tuesday August 01, 2017 @01:54PM (#14903417)
Using unsafe blocks and pointers can silence the borrow checker, but that doesn't mean that code is well defined. For example, using swap (mentioned in the article) with overlapping references would be undefined behaviour.
Re:Rust: Not So Great For Codec Implementing by dbaupp on Tuesday August 01, 2017 @01:09PM (#14902944)
Multidimensional arrays are... completely orthogonal: if the compiler could determine row indices were different, it could determine that slicing indices were too.
Re:Rust: Not So Great For Codec Implementing by kbenson on Tuesday August 01, 2017 @02:12PM (#14903626)
Is that true? Consider variables a=3, b=4, c=7.

With a large single array of 100 elements, but divided into 10 10 element pieces by convention, a slice of elements might be accessed by getting the elements array[10a+b] through array[10a+c]. If we also want a slice from the next segment, that would be array[10(a+1)+b] through array[10(a+1)+c].

With a multidimensional array, of 10 bounds checked segments of 10 elements, we could access by array[a][b] through array[a][c] and array[a+1][b] through array[a+1][c]. If access is bounds checked, or the compiler can prove the values are within the sizes, then the multidimensional version provably has no overlap.

Or am I missing something here?

Re:Rust: Not So Great For Codec Implementing by dbaupp on Tuesday August 01, 2017 @02:53PM (#14904056)
That's true: it probably does make some (probably common) things easier because you get to disjoint slices with simpler indexing conditions, but the problems are equally hard in the general case: the type checker/borrow checker has to understand arithmetic rules like `a != a+1`, and be flow sensitive for things like `if a != b { ... }` and other ways to imply the inequality. (And, it's only a subset of the cases when one wants to slice an array multiple times.)
Re:Rust: Not So Great For Codec Implementing by kbenson on Tuesday August 01, 2017 @02:59PM (#14904137)
> the type checker/borrow checker has to understand arithmetic rules like `a != a+1`

I was actually thinking of this when I wrote, but the opposite case. A lot of compilers already know about that, and exploit it to their benefit (and sometimes the programmers dismay[1]). Overflow semantics matter here.

That said, it's looks like an interesting area for Rust. I'm sure it's already been discussed to death somewhere in the community. :)

1: https://news.ycombinator.com/item?id=14163111

Re:Rust: Not So Great For Codec Implementing by dbaupp on Tuesday August 01, 2017 @03:43PM (#14904580)
Optimisers reason about this, yes, but they don't define the user-facing language model. They take valid code and transform it in to (hopefully) faster valid code[1] and don't generally emit diagnostics or fail compilation, this means that they can do much more guess work and heuristics rather than having to have predictable, reliable or definable behaviour, all of which are useful for humans writing code (having a predictable optimiser is useful too, but difficulty writing any code at all is worse than difficulty optimising the spots where the optimiser isn't hitting performance targets).

Integrating this sort of thing into the type system "properly" (as in, actually part of the language and doing all of what I think Animats wants) basically means going a long way towards full dependent typing.

[1]: behaviour on invalid code, such as C relying on signed integer overflow, is unspecified/undefined. Languages like, say, Rust and Haskell generally try to guarantee the optimiser never gets invalid code by flagging such instances at compile time, unlike most/all C and C++ compilers.

Re:Rust: Not So Great For Codec Implementing by SilasX on Tuesday August 01, 2017 @01:08PM (#14902935)
Is it hard to implement such primitives for distinguishing ownership of subsections of an array? Or to extend Rust to do do that?
Re:Rust: Not So Great For Codec Implementing by geofft on Tuesday August 01, 2017 @02:16PM (#14903673)
It's pretty easy to do it in unsafe code, or to use an existing safe function (implemented internally with unsafe code) like the standard library function split_at_mut to do it.

Here's a doc on how to write such a function yourself, and includes the (actual) source code for split_at_mut:

https://doc.rust-lang.org/nomicon/borrow-splitting.html

As others have mentioned, there's nothing special about the standard library in this regard; you can just write the method yourself and it'll work equally well. (The standard library is moderately special in that it gets to use nightly features because dealing with breaking changes is easy for code that literally lives inside Rust's own source repository, but it's not special in terms of its ability to use unsafe code or anything.)

Re:Rust: Not So Great For Codec Implementing by SilasX on Tuesday August 01, 2017 @02:21PM (#14903724)
I meant primitives that preserve Rust's (safe) ownership model.
Re:Rust: Not So Great For Codec Implementing by geofft on Tuesday August 01, 2017 @02:54PM (#14904083)
Well, you can use the existing split_at_mut if you're worried about getting your hand-rolled unsafe implementation wrong. It's definitely reasonable if you're splitting on rows; it may or may not be what you want if you're trying to get a slice-like thing that splits on columns.

(There's no reason to avoid safe functions that happen to have unsafe implementations. For instance, the standard-library implementation of indexing a slice, as in b"hello"[3], is just a bounds check plus an unsafe dereference.)