News for hackers. Stuff that matters.
Re:Rust 1.37.0 by burntsushi on Thursday August 15, 2019 @05:37PM (#20709926)
That's just one take, and I very much do not agree with it personally. My productivity has only gone up as the tooling and language have improved.

One thing that I am working on figuring right now is the proliferation of dependencies. But that's a"nice" problem to have.

> Just as an example, one of our simulations relies on being able to craft raw UDP packets in order to "impersonate" remote basestations in our lab environment.

I would assume you would do it in Rust just like you would in C. Where's the hang up?

Re:Rust 1.37.0 by nineteen999 on Thursday August 15, 2019 @06:43PM (#20710426)
> I would assume you would do it in Rust just like you would in C. Where's the hang up?

Why assume? If you know it can be done, why wouldn't you provide a pointer to an example? I've done my Google searching and can't find one. There are endless examples of this for C online, I'm just stating that with a cursory search I couldn't find one for Rust.

The hang up is clearly at my end, I'm currently time poor, so now is not a good time for me to be learning a new language where I can't find good examples for what I'd use it for.

Admittedly I need to start at "Hello World" with Rust anyway before I get as far as crafting raw UDP packets. The point I'm getting at is if I didn't know how to program in either C or Rust, I'd pick C for this problem right now because there are clear examples. That's all.

Re:Rust 1.37.0 by burntsushi on Thursday August 15, 2019 @07:10PM (#20710581)
If you don't know how to do it because you don't know Rust, then fine, that's reasonable. (Albeit a strange criticism to lodge from my perspective.)

Otherwise, I would "assume" that you need to use libc bindings over ffi to use whatever udp APIs you would normally use from C. This is how Rust's standard library implements the higher level udp APIs for example: https://doc.rust-lang.org/std/net/struct.UdpSocket.html

But I used a question mark also because this is not something I've done before, so I can't share an example. I wasn't being adversarial. Even my suggestion above could be wrong. That is, I don't understand the essential difficulty here. Perhaps someone else can chime in.

Re:Rust 1.37.0 by nineteen999 on Thursday August 15, 2019 @08:06PM (#20710944)
Sorry, not to be rude but in this case is sounds like you don't understand the problem domain. Perhaps I should have just said "UDP spoofing".

There is a huge difference between creating a generic UDP endpoint like the code you have shown, and crafting a raw UDP socket, where the source IP address is "spoofed", and UDP checksums generated to ensure that the packet looks like a valid UDP packet so that the intermediate network infrastructure like routers, switches and firewalls will pass the forged packet rather than discarding it.

The latter requires quite a bit of extra code. A good example is here:

https://www.binarytides.com/raw-udp-sockets-c-linux/

and last time I looked (month or so ago?) I could not find an equivalent example for Rust. It may be out there but I was unable to locate it. I'm not asking you to do my homework for me, just stating that learning what I need to know in Rust to do the above would take more time than I have available right now. That's not to say I won't figure it out in the future when I have more time.

> If you don't know how to do it because you don't know Rust, then fine, that's reasonable. (Albeit a strange criticism to lodge from my perspective.)

It's not a criticism of the language; what I'm asking may be very possible, I don't know yet. I just know that finding C examples is trivial, Rust examples not so much, so when I had a need for this kind of code recently, I turned to the language where I know it can be done and I could easily find examples.

Re:Rust 1.37.0 by woah on Thursday August 15, 2019 @09:12PM (#20711292)
The specific task is so obscure, that only a language as widespread as C would have an example online for it. So people are confused when you name this as a criteria for selecting a language, since it’s a criteria that most likely only C can satisfy.

Then someone jumped to the conclusion that you were actually saying that Rust doesn’t have enough low level kernel APIs because that would make sense.

Re:Rust 1.37.0 by nineteen999 on Thursday August 15, 2019 @09:31PM (#20711391)
> The specific task is so obscure, that only a language as widespread as C would have an example online for it.

Well you could do it in Python I suppose - here's someone mentioning a solution using scapy, although it likely wouldn't work too well for my particular case due to Python performance.

https://stackoverflow.com/questions/51311741/generating-a-sp...

> So people are confused when you name this as a criteria for selecting a language, since it’s a criteria that most likely only C can satisfy

Rust is intended to be a replacement for C and C++, no? Eventually it will need to be able to do things like this if it is to be effective in that role. It's OK in my mind if it is not mature enough to do these kind of tasks yet, or if it can but I can't find a good reference yet. I wanted to use it for this task, but I selected C instead since it was easy to do that way and I didn't have a lot of time. Not holding it against Rust, it's a much younger language.

> Then someone jumped to the conclusion that you were actually saying that Rust doesn’t have enough low level kernel APIs because that would make sense.

I am almost completely certain I don't understand this sentence the way it is phrased.

Re:Rust 1.37.0 by rixed on Thursday August 15, 2019 @10:02PM (#20711529)
In case you want to give OCaml a try one day, I've done plenty of stuff like that in the past when I needed a (much) faster scappy. UDP exemple here:

https://github.com/rixed/robinet/blob/master/udp.ml

Re:Rust 1.37.0 by nineteen999 on Thursday August 15, 2019 @10:52PM (#20711742)
I'm not really a career programmer or language enthusiast, rather, I'm more of a utilitarian/imperative coder - I write programs to plug gaps in existing infrastructure as quickly, as securely, and as future-proofed as possible, even if it means dumbing the code down somewhat.

So I think my head would explode if I tried to learn a FP language without being able to have months off work to study. In addition to that, my team would kill me, as it's hard enough to find good C/C++ programmers in my city, let alone something like OCaml. Given that we run emergency services infrastructure, unfortunately we have to use more popular languages to ensure that the business doesn't end up with a lot of code written in languages they can't support.

Regardless, thank you very much for the example. It's nice to see that this can be done in other languages even if I can't understand it. Unfortunately too many years of C/C++ have made it hard for me to grok FP code - the C example I posted up before is much more readable to me.

Re:Rust 1.37.0 by ernst_klim on Friday August 16, 2019 @02:59AM (#20712786)
> So I think my head would explode if I tried to learn a FP language without being able to have months off work to study.

What's so hard you find about OCaml or FP in general you haven't had to deal with in your day-to-day imperative programming?

> Unfortunately too many years of C/C++

You mean you find OCaml making your head to explode after years of C and C++?

Re:Rust 1.37.0 by nineteen999 on Friday August 16, 2019 @03:31AM (#20712915)
> What's so hard you find about OCaml or FP in general you haven't had to deal with in your day-to-day imperative programming?

I guess I still find FP very abstract as a concept. I'm starting to like the notion of immutability as a design goal. But then I'm the kind of sicko who enjoys assembly language to some degree, where practically every instruction has a side effect, whether it be assigning a value to a register, or setting a bit in a flag register, or doing I/O whether port mapped or memory mapped.

> You mean you find OCaml making your head to explode after years of C and C++?

Not specifically OCaml, but most FP languages in general. I find the OCaml code posted up a couple of levels to be very hard to understand. But I'm very used to the POSIX interface and honestly programming against much else tends to get me out of my comfort zone a little too much. Shell, Perl and Python I can manage as well because I've been doing them so long, but I don't excel at those, particularly the latter two.

I think the paper referenced in this (https://news.ycombinator.com/item?id=15179188) HN article ("Some were meant for C") captures my mindset very well. Some of us are just a bit broken as programmers.

I also enjoy the ubiquity of C compilers - knowing that I can program in it from anything from a microcontroller to a multicore CPU (even being able to use the same compiler version accross many platforms) is a huge boost.

Re:Rust 1.37.0 by ernst_klim on Friday August 16, 2019 @04:00AM (#20713057)
> I guess I still find FP very abstract as a concept. I'm starting to like the notion of immutability as a design goal.

I'm rather asking about OCaml or SML than about something abstract matters like denotational semantics, system-F or pure lambda calculus.

There are pretty much the same abstract concepts behind imperative programming languages as well: operational semantics, Turing machines, which, I argue, are even more complex.

And if you write OOP, you have to deal with open recursion, covariant/invariant/contravariant subtyping relations, late bindings, polymorphic self type etc etc. There is a book called ``Theory of objects'' by Luca Cardelli (who is guilty for both SML and C++, BTW) if you are interested.

You just don't bother yourself with this theory, and neither should you when programming FP. You don't bother number theory and arithmetic (complex topics) when doing simple addition in your code, right?

> But I'm very used to the POSIX interface and honestly programming against much else tends to get me out of my comfort zone a little too much

Then maybe your starting point should rather be this [1].

(Although this book is rather about system unix programming in OCaml than about language itself. For learning the language its manual should fit pretty well [2], finding a sweet spot between a narrated book and a standard. Other good and more in-depth introductions are [3] and [4]).

There is also a unikernel called mirage os which includes TCP stack and other stuff and gives a good taste of what system programming in OCaml looks like.

https://github.com/mirage

Edit:

> But then I'm the kind of sicko who enjoys assembly language to some degree, where practically every instruction has a side effect, whether it be assigning a value to a register, or setting a bit in a flag register, or doing I/O whether port mapped or memory mapped.

FP doesn't disallow side effects, a usual FP practices just encourage to deal with them explicitly. There are low-level FP languages like ATS [5] (though I discourage you to look at it since it's unnecessarily complicated and could demotivate you), F* [6][7] and even Coq to some degree [8] which allow to deal with effects like allocations and state just fine (and prove invariants).

[1] https://ocaml.github.io/ocamlunix/

[2] https://caml.inria.fr/pub/docs/manual-ocaml/

[3] http://dev.realworldocaml.org/

[4] http://ocaml-book.com/

[5] https://www.youtube.com/watch?v=zt0OQb1DBko

[6] https://fstarlang.github.io/lowstar/html/Introduction.html

[7] https://fstarlang.github.io/lowstar/html/LowStar.html#memory...

[8] https://www.microsoft.com/en-us/research/publication/coq-wor...

Re:Rust 1.37.0 by nineteen999 on Friday August 16, 2019 @04:47AM (#20713292)
Thank you very much for all the time taken to collate all those links. I hope one day I will have the headspace, time and patience to absorb all that.
Re:Rust 1.37.0 by woah on Thursday August 15, 2019 @09:53PM (#20711499)
Re:Rust 1.37.0 by nineteen999 on Thursday August 15, 2019 @10:07PM (#20711544)
Thanks for this, those do look very helpful.
Re:Rust 1.37.0 by burntsushi on Thursday August 15, 2019 @09:22PM (#20711349)
Yeah, I mean, as I mentioned, I hadn't done it before. I feel like I've put an appropriate amount of uncertainty in my comments, but I still feel like you're punishing me for it.

In any case, I don't see anything particularly interesting in that C code. It's just shuffling data around and using some C functions, all of which is pretty easy to do in Rust. With that said, I do agree that translating that code would be non-trivial for a Rust beginner and would require getting pretty comfy in the language first. You'd likely also have to redefine any structs that you need in Rust (or use a tool like `bindgen` to generate bindings to, e.g., netinet/udp.h for you). But still, as an experienced Rust programmer, I think that C example would be enough for me to tackle this particular problem. So this might just be a case of being unfamiliar with the language, which is totally fair. There are going to be far less resources for Rust when compared to C.

Re:Rust 1.37.0 by nineteen999 on Thursday August 15, 2019 @09:46PM (#20711457)
Thanks for the advice regarding bindgen etc, I will look into that when I have more time. Generating the compatible structures (if Rust doesn't have them already) is one part of the problem, the other is really just the sendto() and socket() calls which as I mentioned are basically syscall wrappers for the most part. As per my other email, if I was writing this program in Rust, I'd prefer to use Rust all the way down rather then calling into C. Otherwise it's a Rust wrapper around C facilities. It should be possible do it all in native Rust code as far as I am concerned.

As per my other message, I apologize if I have offended you, it certainly wasn't my intention, I was just trying to clarify the use of the raw IP API, as opposed to the higher level solution which wouldn't work for our simulator.

Re:Rust 1.37.0 by burntsushi on Thursday August 15, 2019 @09:51PM (#20711491)
> It should be possible do it all in native Rust code as far as I am concerned.

Yeah, it would be, but Rust's standard library doesn't provide any way to invoke syscalls. You either need to go through libc's syscall wrapper, or write Assembly. I used the former to get access to faster directory traversal: https://github.com/BurntSushi/walkdir/blob/ec33af7f9b25afefc... (The rest of that code in that module may be interesting to you, since the API for getdents is pretty interesting and very C-like. Buttoning it up behind a safe API that can never be misused took a bit of thinking.)

Besides, Rust's standard library, on Linux at least, currently requires libc anyway. So you're already going to be linking to it.

Re:Rust 1.37.0 by nineteen999 on Thursday August 15, 2019 @09:41PM (#20711438)
My apologies - I don't mean to come off as adversarial either. And as I clarified in my other message, I should have just said UDP spoofing, so I probably contributed to the confusion.

> I would "assume" that you need to use libc bindings over ffi

I wonder about whether that is even necessary. socket() and sendto() are basically C library wrappers around system calls for the most part. Surely the Rust standard library already has equivalent wrappers, or will someday. There's no real reason I can think of that I'd need to FFI into C unless there is some limitation of the current Rust standard library.

Re:Rust 1.37.0 by burntsushi on Thursday August 15, 2019 @09:48PM (#20711472)
No worries.

You can use libc::syscall to make syscalls in the normal way, and that's technically ffi. Otherwise, to do raw syscalls, AFAIK you need to write some Assembly to do it. Running an assembler and linking it into a Rust program is pretty straight-forward. It would be nicer if one could use inline assembly (and let the compiler handle it for you), but that hasn't been stabilized yet.

The topic of whether to build a Rust standard library without libc is definitely one that has gotten some attention, and some folks have made some progress on that front, but AFAIK there is no serious ongoing project or effort that attempts this. Some folks definitely desire it though, at least for Linux. (In other platforms, like macOS and Windows, raw syscalls aren't a stable interface, so you kind of need to use the system libraries.)