News for hackers. Stuff that matters.
Re:Linux kernel in-tree Rust support by daenz on Saturday July 11, 2020 @08:30PM (#23807504)
6 months ago, as I hopped on the Rust train, I told a friend of mine that I'm convinced that Rust will "eat the world" like Python did[0]. These moves to see Rust enter the Linux kernel only prove to me that this pace is starting to pick up.

0. https://www.techrepublic.com/article/python-is-eating-the-wo...

Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @09:06PM (#23807719)
This is a deeply weird comparison.

Insufferably lazy "eating the world" headline aside, Python is incredibly popular because its runtime performance is "good enough" for a huge fraction of applications, its safety guarantees are leagues better than C/C++, and it's deliberately simple to write (it was designed in part as a pedagogical language).

Rust's performance is leagues better than Python's, it's safety guarantees are better by degrees, and it is substantially harder to write than Python, not just because it requires a more sophisticated understanding how how programs are constructed and verified, but also because it requires developers to think carefully about memory management.

I fully expect Rust to "eat the world" of C/C++ code over the next 10 years, and look forward to a time where we'll look askance at kernel code still written in C. It will no doubt be important in a bunch of performance-critical settings (AAA gaming, browsers, network infrastructure) as well.

But Rust will never "eat the world" of general applications (that is, the overwhelming majority of applications), because Python is good enough across all axes for these applications. It would be engineering malpractice for most companies to rewrite working Python applications in Rust. That was not the case for C-to-Python (or even PHP-to-Python) rewrites.

In case this isn't obvious to anyone reading this thread --- though I'd be surprised --- the reason Rust is in the Linux kernel, while Python isn't, is that Rust was designed to be interoperable with kernels and browsers, and Python, like most other languages that have ambitious runtimes, was not. It's not an indication of Rust's popularity, but rather just Rust doing something it was designed to do.

(Rust is plenty popular.)

Re:Linux kernel in-tree Rust support by daenz on Saturday July 11, 2020 @09:20PM (#23807797)
I think you're misunderstanding the comparison to Python. I am saying Rust will eat the world in the space that it fits in, which is still relatively large, in the same way that Python did, in the space that Python fits in. Not that Rust will take over very high level languages too.
Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @09:23PM (#23807816)
The reason I was moved to comment is that I think it's actually not that large a space (the one that Rust plays in). Instead, I think it's just that the applications in that space loom large in our consciousness, because we all use and pay attention to kernels and browsers. And I expect the space of applications "languages like Python" play in to grow over time, and Rust's to shrink, not just because computers will get faster but because the runtimes for those languages will get cleverer (see, for instance, what's happening with Javascript).
Re:Linux kernel in-tree Rust support by nilkn on Saturday July 11, 2020 @09:51PM (#23807988)
I would actually like to make the exact opposite prediction.

A generation of developers grew up on Python, Ruby, etc., rather than C, and they were all brought up on this notion that performance doesn't matter much anymore because computers keep getting faster. But the problems we're solving with computers keep getting harder (in terms of the amount of computation required).

These days, I can comfortably state that I prefer Rust over Python, even for simple programs. I prefer it because the type system is much more useful than Python 3 with mypy, the performance is extraordinary out of the box, the language is often just as expressive, and I rarely have any actual issues with the borrow checker. I also prefer it because I've had far too many experiences with Python where supposedly performance didn't matter... until it did and Python was no longer appropriate.

I think we're entering into a generation of languages where things like Rust are going to start really seriously displacing C++, and as more people learn them they're even going to start displacing supposedly easier languages. Right now, few folks are going to learn Rust as an alternative to Python. But once they're exposed to it for other work and realize how productive they are, there's just going to be less of a need to reach for Python over time.

Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @09:53PM (#23808000)
What is the thing happening in computing making the largest demands on compute performance? Data science and ML. And what's the lingua franca of data science and ML? Not even Java or Go, the managed compiled languages; no, it's Python.†

You can see the phenomenon I'm talking about happen right now, in every HN thread where we push against the ocean trying to keep Electron from being a thing. What's going to happen first: Electron's runtime will become clever enough that its performance is tolerable even to nerds, or most Electron applications being rewritten in a language that forces programmers to manually manage memory?

It's interesting, because Rust will probably be a part of the story that makes things like Electron tolerable! But the point isn't that Rust is a doomed language (it's not, it's going to be durably important), but rather that it's not going to be the language most programmers work in.

† It's not pure Python, of course, but that's part of my point.

Re:Linux kernel in-tree Rust support by nilkn on Saturday July 11, 2020 @10:13PM (#23808128)
> What is the thing happening in computing making the largest demands on compute performance? Data science and ML. And what's the lingua franca of data science and ML? Not Java or Go, the managed compiled languages; no, it's Python.

While ML might be the most demanding area in terms of computation, it's also a very small part of what developers in general do. My claim/prediction is more based on the notion that everything is becoming more compute-intensive.

Where I work, we've probably incurred many, many millions of dollars of expenses (primarily hardware) due to the choice to write a lot of software in scripting languages over the years. We made that choice because C++ was so unwieldy in comparison that we figured we'd end up spending the same amount on C++ but in different ways. If we were starting from scratch today, I doubt we'd use scripting languages like Python for hardly any of our software, because there are choices available now that just didn't exist in the past.

I'm not saying there's no place for something like Python, nor am I saying existing Python stuff should be rewritten. I think Python is going to continue being an extremely popular and important language. My claim/prediction is more that in the future we'll increasingly see Python chosen less for completely greenfield projects because there are now options that are high-level, well-designed, and extremely fast, and at least in my experience that has a lot of business value that is going to be harder and harder to ignore over time as it becomes more obvious.

Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @10:49PM (#23808339)
Candidly, there once was a vocal subculture of developers that would aver as to how much more comfortable they were in C than in Python (or Tcl or Perl), and, I'm told, in the long long ago there were them that said the same thing about assembly language. As an engineering manager, your job, if done well, would be to mollify those developers without conceding any ground to them, because the higher-level language was practically always the better option.

People will burn money on compute no matter what language things are built in; they'll ship on Fargate instead of EC2, on-demand instead of reserved instances, RDS instead of running their own Postgres, Postgres instead of just using Sleepycat, anything at all instead of running their own iron, and on and on down the list.

(All of this is out the window if you're shipping a browser! Or a hypervisor! Or an OS kernel! Or a new router! Or a AAA game!)

Most of the time, the most expensive thing you can burn is an hour of programmer time. And, with your fluency in Rust and other languages, you must see immediately why someone would believe Rust is more time-consuming than, say, Java, let alone Python.

Paul Graham got this part, at least, right in "Beating The Averages", when he noted how much more worrisome a competitor would be if their job reqs were for Perl programmer than if they were for C developers. The Perl company is actually getting lots of stuff done.

Re:Linux kernel in-tree Rust support by nilkn on Saturday July 11, 2020 @11:08PM (#23808464)
You have two premises here, one explicit, one implicit:

(1) [explicit]: a higher level language is practically always the better choice.

(2) [implicit]: Rust is not high level (or at least not compared to Python).

Our biggest disagreement is with (2). Rust is actually higher level and more expressive than Python in many ways. Of course, it also exposes some lower level concepts like lifetimes and ownership that Python hides.

Your comparison doesn’t apply because there isn’t a simple hierarchy into which Rust and Python fall. Each language is both higher- and lower-level than the other in a number of ways.

Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @11:14PM (#23808498)
The word "expose" is doing a lot of work there.

I've written a lot of Python (thankfully not so much anymore), and Rust is part of my day job. I am struggling to think of a circumstance in which something would be easier to write in Rust than in Python, factoring out "performance" --- just comparing the languages themselves. Maybe, you can provide examples, and we can see if we're working from the same premises.

Re:Linux kernel in-tree Rust support by akiselev on Sunday July 12, 2020 @12:46AM (#23808899)
> I am struggling to think of a circumstance in which something would be easier to write in Rust than in Python, factoring out "performance" --- just comparing the languages themselves.

Most Python backend projects I've worked on would have been better off written in Rust (had it existed) the same way most of the Javascript projects I've worked on would have been better off written in Typescript.

Maybe it's just the projects I've worked on (or their size) but maintenance and refactoring to deal with growth of or changes in business requirements were by far more time consuming than actually writing code. The amount of time lost on onboarding and writing trivial tests that could have been handled by the Rust compiler dwarfs true R&D time, let alone time spent debugging bugs that could have been prevented by encoding business logic into the type system.

At my current job I often still pull out Python to prototype logic or Typescript to spin up a quick and dirty intranet app but for the stuff that needs to last, just work, or be maintained into the future, I often find myself yearning to use Rust. The last obstacles are organization talent pool and Rust's ecosystem.

Re:Linux kernel in-tree Rust support by hu3 on Sunday July 12, 2020 @08:18AM (#23810786)
As much as I'd love to use more performant APIs and programs that crash less on my daily basis, I don't see Rust even close to overtaking Python.

For starters, memory management in Python is trivial while in Rust the developer has to learn a whole new set of skills in the form of borrowing, lifetimes and ownership.

And for large projects where Python's limitations hamper team coordination, GC'd languages such as C# and Java offer much softer learning curve and a mature ecosystem. Even when performance is critical, projects get by just fine with using Java like in High Frequency Trading shops.

Perhaps we shouldn't settle for Rust but refine the idea in a new, more ergonomic, language with focus on achieving the same but with lower cognitive load. Faster compilation times wouldn't hurt too.

Re:Linux kernel in-tree Rust support by akiselev on Sunday July 12, 2020 @07:32PM (#23815639)
I rarely use Rust for the performance or memory safety - it's a nice perk that allows for data structures and algorithms I wouldn't otherwise use but it's not the main reason I reach for it. I use Rust for its type system including traits, phantom marker types, affine types, sum types, exhaustive pattern matching, and so on. Once you get used to traits and composition over inheritance, it allows writing very reusable code that enforces a significant fraction of logic at the compiler level and scales much better with code and team size (modulo dependency hell, which I think is language agnostic but a work in progress in Rust). Rust's model of wrapping unsafe code in safe types also makes working with the world of C/C++ libraries much more pleasant once you get the hang of FFI.

Plenty of other languages have these features but not quite in the right combination or their communities are too small and they haven't caught on yet. Rust is, in my opinion, at an organizational and technical sweet spot for large projects.

Re:Linux kernel in-tree Rust support by nilkn on Saturday July 11, 2020 @11:15PM (#23808505)
> The word "expose" is doing a lot of work there.

I do not believe this is a statement that I can reply to in a meaningful way. You definitely believe Rust is much harder or more complicated than I think it is, and that’s basically where we disagree.

Re:Linux kernel in-tree Rust support by entha_saava on Sunday July 12, 2020 @12:27AM (#23808813)
Claiming rust is as expressive as Python/JS/Kotlin for non-fp-nerds is precisely what leads to rust evangelism strike force memes.
Re:Linux kernel in-tree Rust support by nilkn on Sunday July 12, 2020 @12:46AM (#23808897)
There’s not any particular way I can respond to this, because there really isn’t any substance except to attempt to link me to something you find distasteful/annoying. I use both languages professionally on a daily basis and it is on that basis that I make my claim.

Here’s one way I could respond. There are numerous things in Python that I find more difficult than in Rust. For instance, I think async/.await in Rust is generally easier than asyncio is in Python, and in the former case the compiler is quite good at detecting various concurrency bugs whereas you’re entirely on your own in Python.

It’s far easier to write multicore programs in Rust than Python thanks to the lack of a global interpreter lock.

Python makes extensive use of runtime exceptions for basic error handling, which I find very exhausting to reason about and account for during development; tools like mypy are also unable to help there at all, and I must rely on the questionable quality of documentation or just scavenging through code.

Pythonic things like None-equivalent values often make it impractical to use Optional values as return types in idiomatic code, thus pressuring you to use more exceptions in places where one would just use Option or Result in Rust.

Python has poor support at best for sum types, which I tend to use extensively for almost every problem space.

Python does not support pattern matching, though some support for this has been proposed for a future version of the language.

Handling dependencies generally is much easier in Rust as well thanks to cargo.

Perhaps you think these are all just the concerns of “FP nerds,” but for me these differences often mean I can write correct, high quality backend services in Rust faster than I can in Python — and the result is often 10x-100x faster with no optimization work whatsoever. That seems worth the relatively small amount of time I spend worrying about lifetimes or ownership or the fact that I use map/filter/iterators instead of list or dictionary comprehensions.

Re:Linux kernel in-tree Rust support by comex on Sunday July 12, 2020 @05:25AM (#23810091)
> You can see the phenomenon I'm talking about happen right now, in every HN thread where we push against the ocean trying to keep Electron from being a thing. What's going to happen first: Electron's runtime will become clever enough that its performance is tolerable even to nerds, or most Electron applications being rewritten in a language that forces programmers to manually manage memory?

Browsers already have an immense amount of effort poured into making them fast. They will get faster, but not by much.

But then, JavaScript was never the problem with Electron in the first place. Or at least, not the fact that UI code is written in JavaScript. After all, a lot of the people complaining about Electron (me included) are Mac users pining specifically for native Cocoa apps. Which, up until recently, were all written in Objective-C – a language which is much slower than JavaScript! Even Swift is not necessarily faster.

Rather, from a performance perspective, I think the biggest problems are (1) the DOM being a terrible abstraction to design user interfaces on top of, and (2) the web forcing you to use JavaScript for everything rather than letting you drop down to native code for performance-critical bits (other than what is implemented by the browser itself).

Re:Linux kernel in-tree Rust support by jariel on Saturday July 11, 2020 @10:21PM (#23808178)
"I can comfortably state that I prefer Rust over Python, even for simple programs"

That's fine but that will never be generally true because Python is a high level language that will always have 10x more developers using it, including your Data Science prof, the high school kids learning 'basic programming', and all the AI researchers.

And the tons of devs doing stuff for which Python just makes more sense.

Re:Linux kernel in-tree Rust support by nine_k on Saturday July 11, 2020 @10:39PM (#23808279)
I don't think it's appropriate to call a language with an advanced and useful type system, expressive syntax out of box, and reasonable macro support a "low level" language.

Python, in comparison, lacks a lot. But it has a very low barrier of entry: the syntax is English-like, you can start doing something with very little knowledge of advanced features, and everything is easy to inspect and play with in a REPL.

Re:Linux kernel in-tree Rust support by jariel on Saturday July 11, 2020 @10:45PM (#23808310)
"advanced and useful type system, expressive syntax out of box, and reasonable macro support"

Those things are nice, but don't have anything to do really with the question here.

Rust is designed to be 'close to the metal' and due to that fact, and a few other things - it's an 'advanced' programming language.

Javascript, Python (and PHP at least today but that's changing) will have 10x a greater user base than Rust ever will.

In the Enterprise, Java and C# are going nowhere.

Rust will be the domain of only a specific set of software developers, and they will share that space with C++.

Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @10:52PM (#23808352)
Respectfully, I feel like you're dancing around what Python has that Rust does not. It's not the "English-like syntax" (with its invisible syntax errors). It's that Python, like most modern languages, manages memory automatically, and Rust does not.

Rust doesn't for good reason! Not just zero-cost abstraction, but also, because for the applications it's designed for, it makes controlling memory much easier. There are ways in which writing a Rust program is probably easier than writing a Java program, in Rust's stomping grounds. It's just that it doesn't make things easier anywhere else.

Re:Linux kernel in-tree Rust support by paavohtl on Sunday July 12, 2020 @06:46AM (#23810413)
I don't disagree with your premise, but I disagree with the claim that Rust doesn't automatically manage memory. Rust absolutely does automatic memory management - that's one of the main motivations behind the ownership/lifetime system. For instance, when you create a vector, the standard library will automatically allocate, re-allocate, clone and/or deallocate the buffer when necessary.

What Rust doesn't do is to decide for you how to pass or share that memory between different parts of your program - it just provides you with options. One could argue that this should also be under the automatic memory management -umbrella, and I wouldn't necessarily disagree. But I don't think it's fair to say that Rust doesn't have automatic memory management.

I would also argue that for many simpler programs you can get away with basically paying no attention to memory management. If your program is a simple pipeline which reads some input, applies some processing and produces an output, your code might not look substantially different from any GC-based language. That doesn't mean Rust is an easy language or even a good idea to use for these kinds of programs, but I would argue that a developer proficient in both Rust and Python could write the same (simple, pipeline-like) program in either language with comparable time and effort.

Re:Linux kernel in-tree Rust support by nine_k on Sunday July 12, 2020 @09:01PM (#23816103)
Rust is a high enough level programming language to have a GC as a library. You can go with Rc and Arc all over the place, and have a Python-style ref-counting GC.

OTOH Go is a lower-level, less expressive language with a built-in GC memory management.

To me, the level of a language is the level of abstraction one can comfortably reach with the language, and not being close to the metal or not.

Re:Linux kernel in-tree Rust support by TylerE on Saturday July 11, 2020 @09:26PM (#23807840)
I think nim is the really interesting one to watch here...

Compared to rust it offers, say, 99% of the performance and 70% of the safety while really being almost as easy as python.

Re:Linux kernel in-tree Rust support by entha_saava on Sunday July 12, 2020 @12:29AM (#23808824)
But do you have a HackerNews spamming taskforce that mentions nim in every comment?
Re:Linux kernel in-tree Rust support by nindalf on Sunday July 12, 2020 @07:12AM (#23810515)
Please don’t comment like this. Snarky accusations like this are impossible to respond to and are against the HN guidelines.
Re:Linux kernel in-tree Rust support by JoshTriplett on Saturday July 11, 2020 @09:24PM (#23807822)
As a Rust developer, I completely agree. I'd like to see Rust fully supplant C++ and C, but I don't expect it to fully supplant scripting languages, and I definitely wouldn't suggest rewriting working Python in Rust unless you need to improve performance or scalability, or you're having serious maintainability problems. Let many languages bloom.
Re:Linux kernel in-tree Rust support by correct_horse on Saturday July 11, 2020 @10:05PM (#23808065)
Shoutout to the lovely pyo3 library for letting you incrementally rewrite small parts of your Python in Rust.
Re:Linux kernel in-tree Rust support by remexre on Saturday July 11, 2020 @10:36PM (#23808263)
> Python is incredibly popular because its runtime performance is "good enough" for a huge fraction of application

> But Rust will never "eat the world" of general applications (that is, the overwhelming majority of applications), because Python is good enough across all axes for these applications. It would be engineering malpractice for most companies to rewrite working Python applications in Rust. That was not the case for C-to-Python (or even PHP-to-Python) rewrites.

I dunno, I sort of hope that there'll be good and resource-light application development frameworks in Rust that do "eat the world." I spent most of a summer two years back using an (at that point) 12-year-old machine without X for both my personal and work computing. I did browsing with w3m+imlib2, images and PDFs with fbi and fbpdf (iirc), and fbterm (and tmux) as a terminal emulator. That was leagues more responsive than the firefox/eog/zathura/konsole setup I reverted to once the school year started (and I needed to run webapps again).

If you're familiar with https://en.wikipedia.org/wiki/Wirth's_law I think it applies really strongly to this case, and it's gotten remarkably worse as programmers have switched to making general applications with Python and Electron at the same time as single-threaded clock speeds have slowed down.

Re:Linux kernel in-tree Rust support by jariel on Saturday July 11, 2020 @10:18PM (#23808159)
Python is high level 'tool' - accessible to grad students from various fields looking to do 'some things' with 'data'.

Rust is effectively for 'software developers'.

Also: ", because Python is good enough across all axes for these applications.", no, Python is good for a variety of things, but not remotely for every thing. Javascript/Typescript, C#, Java and its offspring, and probably Golang, are going nowhere. And of course Swift.

Finally: incumbency is more powerful than anyone expects. If C/C++ 'works' in any given scenario, it will be difficult to replace. So many embedded systems, so much hardware out there.

Think of how much C++ Code Adobe has invested in. Think of the massive array of sophisticated Autocad products all written in C++. They are not going to be refactored. The native parts of the JVM, Oracle DB, the list goes on.

I predict in 10 years 'greenfield' projects might favour Rust, but many won't.

Re:Linux kernel in-tree Rust support by Reelin on Saturday July 11, 2020 @10:39PM (#23808276)
> incumbency is more powerful than anyone expects

I agree, and would also point out that the competitors aren't static targets. D seems to be well on its way to getting a borrow checker. Well written C++20 looks nothing like C++98. Etc.

Re:Linux kernel in-tree Rust support by blandflakes on Saturday July 11, 2020 @09:25PM (#23807828)
Yes! The number of settings where $RUST_COMPETITORS are applicable is simply smaller than the number of settings where a high-level, dynamic or at least garbage-collected language is applicable. It's hard for me to imagine having a real justification for building services in Rust given the relative productivity of those high-level languages.

This is basically the reason I haven't picked it up for any personal projects, either. I admire the goals, but my interests don't call for a C++ killer.

Re:Linux kernel in-tree Rust support by person_of_color on Saturday July 11, 2020 @09:16PM (#23807774)
Ok, you've convinced me. What's the best way to learn Rust in 2020?
Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @09:17PM (#23807778)
The Rust book, which is free online, an idea of a simple program you might have written in some other language, and a text editor.
Re:Linux kernel in-tree Rust support by fctorial on Saturday July 11, 2020 @09:39PM (#23807925)
> Python is incredibly popular because its runtime performance is "good enough" for a huge fraction of applications

Is there anything slower than pure python?

Re:Linux kernel in-tree Rust support by devenblake on Saturday July 11, 2020 @10:11PM (#23808110)
Interpreted BASIC, of which (in my opinion) Python is the spiritual successor.

I took an online class in college on Python and one of the assignments required the program to scan a plaintext wordlist for a word. I hesitantly made Python load in the entire wordlist and split it by newlines into values in a list, before something like "if word in wordlist". I fully expected my computer to grind to a halt because the interpreted language was loading in a four megabyte textfile, and I expected to have to make a much more complicated function to scan to see whether the word was in the list. I ran it and it ran in less than a second before printing whatever I put to signal that it worked. I didn't believe it so I ran it again with the same result.

Python is like BASIC if it was stable enough to be mission-critical and fast enough to work where it needs to. My previous experience with interpreted languages was with TI-BASIC and it blew my mind that my laptop was powerful enough to do something that fast. Sure, it's slow, but if you really need speed you're better off writing something in something like C or FORTRAN (or Rust now) anyway.

Re:Linux kernel in-tree Rust support by t-writescode on Saturday July 11, 2020 @09:59PM (#23808036)
Ruby.
Re:Linux kernel in-tree Rust support by riwsky on Saturday July 11, 2020 @10:01PM (#23808044)
yes: human beings.

For basically everything we can feasibly ask python to do, it can do that thing faster than we could on our own; few are the things that we need done faster than that.

Re:Linux kernel in-tree Rust support by cheez on Saturday July 11, 2020 @09:52PM (#23807994)
I'm pretty sure that's exactly what he meant.
Re:Linux kernel in-tree Rust support by baby on Saturday July 11, 2020 @09:07PM (#23807727)
Javascript did this.
Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @09:08PM (#23807731)
Sure, I mean, in my head, Python, Ruby, and Javascript are all lumped together (though Python appears to be by far the most popular of them in commercial settings).
Re:Linux kernel in-tree Rust support by Aeolun on Saturday July 11, 2020 @09:24PM (#23807821)
I’ve really only seen python used in academics and data processing.
Re:Linux kernel in-tree Rust support by tptacek on Saturday July 11, 2020 @09:32PM (#23807873)
In the almost 4 years I spent doing security for startups, Python was overwhelmingly the most popular language, not just among clients but among every startup we talked to. That's not counting data science teams.

(I'm not really a Python fan, for what it's worth.)

Re:Linux kernel in-tree Rust support by AmericanChopper on Saturday July 11, 2020 @09:46PM (#23807961)
My recent experiences with startup consulting have involved a lot more new node projects than new python ones. Though I also lump those three languages together as all doing mostly the same thing.
Re:Linux kernel in-tree Rust support by baby on Saturday July 11, 2020 @09:52PM (#23807990)
They are close contestant for sure, but don't underestimate the popularity of Javascript among developers: https://madnight.github.io/githut/#/pull_requests/2020/2
Re:Linux kernel in-tree Rust support by nitrogen on Saturday July 11, 2020 @09:35PM (#23807898)
There are more niches, industries, and developer cultures than any one of us can fathom.
Re:Linux kernel in-tree Rust support by wolco on Saturday July 11, 2020 @10:25PM (#23808199)
PHP to Python rewrites? That seems more foolish then Python to Rudt. I have seen a few Python to PHP rewrites.

Python is taught in school instead of java now which took over c/c++ as the default language.

Re:Linux kernel in-tree Rust support by BlahGod420 on Saturday July 11, 2020 @08:39PM (#23807574)
I've been keeping an eye on crates.io stats - it's definitely booming. ~50 new crates a day (some of them are total nonsense/tutorial work/etc.).

I keep wondering about whether the hype is justified - I must be slowing down because I find myself hanging for long periods of time on borrow checker errors. One of the errors has stopped my progress dead for a week now, I swear it worked a week ago and then Rust decided that a borrow I was doing was no good.

I've also heard the borrow-checker is a common hold up for new Rust developers, so I keep slogging on.

Re:Linux kernel in-tree Rust support by coldpie on Saturday July 11, 2020 @08:48PM (#23807621)
I'm also relatively new to Rust, just about to ship my first ever project written with it. Borrow checker errors are definitely tricky. In my experience, it usually indicates that you've done something "the wrong way," which means you need to re-think it and architect it somehow else. There's often not a quick fix (unless you just made a typo or something, of course). For example, think about who owns the data in question, why it's trying to be stored in two places at once, and how you can re-arrange your data structures or code flow to avoid that condition. It gets harder to understand once lifetimes get involved, I still have to dig into the book to understand the ins & outs of that, and its syntax. Once you get the hang of writing code in a Rust-friendly manner, borrow checker errors mostly just don't happen, because you wrote your code "the right way" to begin with.

I hope I described that well. It's a fascinating language, and now that I'm on the far side of the learning curve, I find it very pleasant to use. I urge you not to give up.

(Have you read through and written, built, and played with the examples in the book? I found it extremely helpful to follow the exercises, and intentionally break them, to test my understanding. https://doc.rust-lang.org/book/ )

Re:Linux kernel in-tree Rust support by BlahGod420 on Monday July 13, 2020 @05:07AM (#23818722)
You're absolutely right, I think your explanation is well written. I did get past the error that stopped me for a week, and yeah it is a case of tackling the same workload but in a different way (I guess 'the right way'). What surprised me is that the solution came to me while I wasn't looking at the compiler or source or anything, but was more of an "ah ha!" moment while walking around. This indicates to me that maybe I will eventually come to understand the borrow checker. Also that book is fantastic, I find myself regularly reading/re-reading it as well as "Rust By Example". Also the std docs are great to work through when I am trying to figure out a way to accomplish something.
Re:Linux kernel in-tree Rust support by _bxg1 on Saturday July 11, 2020 @08:52PM (#23807646)
1) Use rust-analyzer if you aren't already. It decreases the time it takes to see compiler/borrow error results by about 10x. This helps enormously when you're banging your head against something, taking guesses to see what might work.

2) It's often better to slow down and think through what's going on. Make sure you have a good understanding of pointers, allocation, ownership, etc, and if you've been fighting with something for more than 10 minutes see if you can actually understand why it's upset. Rust makes it easier not to accidentally mess these things up, but you will still be chronically frustrated if you don't have a good grasp on them.

3) I've found that the more I use it the more I get an intuition for what fixes certain errors. I don't feel great about fixing something without fully understanding it, but for certain common cases it can really help with the problem of iteration speed.

Re:Linux kernel in-tree Rust support by staticassertion on Saturday July 11, 2020 @08:47PM (#23807617)
A week? Might make sense to refactor, or just clone the data? Or ask the subreddit, discord, or whatever place you want for help.

I haven't run into a borrowchecker issue that took me more than a few seconds in a few years. The trick is I'm not crazy about avoiding cloning - Rust is still super fast. The curse of Rust making cloning explicit, and move by default being cheap/ memcopy, is that you often feel like "if only I could express it" and that problem seems to get worse the better you are, and then for me I hit a point where I stopped caring and just got shit done.

It's easy to remove clones later in my experience, once benchmarks show it matters.

Re:Linux kernel in-tree Rust support by estebank on Saturday July 11, 2020 @09:48PM (#23807969)
Rust has one curse (it has many, but this one is critical): inefficient code is generally visible. Experienced developers hate to notice that their code is inefficient. They will balk at seeing `Arc<RefCell<T>>`, but won't bat an eye at using Python. I know because I have the same instinct! This makes it much harder to learn Rust for experienced devs because they go from the "simple code that will work but is slightly inefficient" and in an effort to improve it they land squarely in parts of the language they haven't yet developed a mental model for.
Re:Linux kernel in-tree Rust support by zozbot234 on Sunday July 12, 2020 @07:54AM (#23810675)
The most important mental model to develop is a model of what can and cannot be feasibly optimized. Sometimes there really is no alternative to stuff like Rc<RefCell<>>, Arc<RwLock<>> or Arc<Mutex<>> - because you'll be working with pieces of data that have to be mutated from multiple places, and don't even have a well-defined lifetime so have to be reference counted. In that case, you should write a comment to that effect and move on. It's still good to see that RefCell<> and the like, because shared mutability is known to be a source of logical bugs. It's a bit like a low-key `unsafe`.
Re:Linux kernel in-tree Rust support by BlahGod420 on Monday July 13, 2020 @05:16AM (#23818785)
Yes, a week (no shame, damnit). The case was that I couldn't understand what the problem even was. I understood that there can be only one owner, but I didn't see how what I was doing was a problem. I literally looked at every error message it gave me and was like "so what?"

I think Rust initially suggested I implement the Copy trait or something, and so I had to brush up on traits and all that (which I did), only to find that my types can't have traits because vec<T> doesn't implement the Copy trait. So I abandoned that.

The problem turned out to be that I needed a minor refactoring. I think the issue eventually turned out to be that I was mutably iterating through a list reading from TCPStreams, and then trying to write back to those streams. So it was like a read_object_from_stream(stream) on each element in a vector, and then the next line was a broadcast_to_all_streams(message) (I'm psuedocoding).

The fix for me (when I finally came to understand what was happening) was to read_object_from_stream(stream) in to an intermediary vector (I named it cache), and THEN iterate through the intermediary and broadcast each item.

In my notes I paraphrased it as "Instead of doing a readwrite, I separated my algorithm in to a READ and then a separate WRITE", because the way things were written I was double borrowing.

Re:Linux kernel in-tree Rust support by estebank on Saturday July 11, 2020 @09:42PM (#23807945)
Could you share what the problem you're having is? People in https://users.rust-lang.org/ and https://reddit.com/r/rust are more than happy to help, and if I could peek at a repro case running in https://play.rust-lang.org/ I could try to help you too (and potentially fix the diagnostic in the compiler). Also, depending on what the problem is, you might have an easier to understand error in newer versions of the compiler, which is why I recommend newcomers to use nightly and update often: not because they'll use advanced nightly-only features but because we merge diagnostics improvements aimed at newcomers every week.

Also, don't get too discouraged. If you can avoid doing things that require advanced lifetime checking then you can gain enough experience with the rest of the language letting you focus only on that part of the language at a later time. Also, I can confidently tell you that at some point it clicks and you start anticipating things that can cause issues.

Re:Linux kernel in-tree Rust support by BlahGod420 on Monday July 13, 2020 @05:25AM (#23818824)
Thank you for your response. I did figure out the problem after I took the time to slowly and methodically understand what the compiler messages were saying - that being said I was looking for places to ask for help if another week went by and I was still stuck so your resources are in my favorites list now.

As I wrote in another post, I came up with the solution while out walking around and it was like an "Ah Ha!" moment, I think things are finally starting to click a bit more. I purposefully chose to write a chat server/client because it lets me work through language features one at a time. There is some basic printing/reading of console input (though I did go full blown Cursive), some networking, threading, and lots of string manipulations. I am not sure when lifetimes will come in to play, but I have started reading about them and I try to think about the lifetimes of my objects when I write.

Re:Linux kernel in-tree Rust support by daenz on Saturday July 11, 2020 @08:53PM (#23807655)
Don't get too hung up on fancy structs containing borrowed data with complicated lifetimes. Zero-copy operations via references are an optimization that you can often forego unless you need really high performance code.
Re:Linux kernel in-tree Rust support by JoshTriplett on Saturday July 11, 2020 @08:57PM (#23807679)
I remember it taking me a solid month or two (of side-project experimentation, not daily work) before I felt like I'd really internalized the right mental model for ownership and borrowing.
Re:Linux kernel in-tree Rust support by BlahGod420 on Monday July 13, 2020 @05:26AM (#23818832)
I'm about a month in. I decided to write a chat server/client to learn Rust by. Multiple side-projects sound like it would also be advantageous. Thanks for your comment.
Re:Linux kernel in-tree Rust support by steveklabnik on Saturday July 11, 2020 @09:05PM (#23807718)
Please don’t hesitate to ask for help! The forums and discord are there as a resource.
Re:Linux kernel in-tree Rust support by nilkn on Sunday July 12, 2020 @11:35AM (#23812101)
Out of the box, Rust encourages almost everything to be stack allocated, imposes a read-write lock on every value (i.e., you can have multiple immutable readers, but only one mutable writer at a time), and uses move semantics for variable assignments (i.e., values aren't copied but rather ownership of the value changes from one variable to some other variable).

Sometimes you need to drop one or several of these properties. It's very easy to do that -- Rust just makes you be explicit about it. If you want something to be heap allocated, you can use a Box/Rc/etc. If you want a value shared among multiple writers, there are several options; a common one is Arc<Mutex<T>>, which is also thread-safe. If you don't want move semantics, you can just copy the value using .clone().

A few lessons to take away, though:

* Because of that read/write lock I mentioned, it follows that you generally want to avoid taking a mutable reference to an entire data structure, because doing so effectively puts that writer lock on the whole thing. If you can take a mutable reference only to the piece you actually care about, that can help out a lot in some cases.

* Rust makes you feel like you should never clone, but really it's up to you. Exercise basic judgment that you'd use in any other language, and don't let perfect be the enemy of the good. In many cases it's probably not nearly as expensive as you might feel it is, and sometimes it'll just be optimized away in the end anyway.

* Try to stick with structs and data structures that own their data as your default, deviating when it makes sense. I've seen people try to avoid ever cloning values by having many structures only contain references, which leads into complicated lifetime management that they weren't prepared for (and that may not even be possible or make sense).

* Take a moment to understand the various wrapper types (I like [0]). Intuitively, Rust gives you a set of guarantees that you can ask for, and you just pick the guarantees you need and compose them together.

[0] https://manishearth.github.io/blog/2015/05/27/wrapper-types-...

Re:Linux kernel in-tree Rust support by incompatible on Saturday July 11, 2020 @09:08PM (#23807728)
Generally a language with automatic garbage collection is easier to work with. Rust can still be useful in areas like the kernel.