I don't really get this attitude, it means that nothing new will ever overcome anything that's been established.
> Anything that can be done easily in C or C++ will need to be easier in Rust for everyone to move
You don't even need folks to move from C. Rust has had lots of success when it comes to folks completely new to systems programming learning it through Rust. Predominantly python/ruby/whatever shops are using Rust because they need a fast language, but don't want to deal with safety issues.
> Blog posts wont pull me away from C, tooling and docs will.
Then you are not the right audience for this blog post :) I find that such blog posts are extremely helpful in convincing people who have a choice between starting to use C and starting to use Rust, not folks who are already invested in C or C++. But I have seen such overviews to have impact on invested C/C++ programmers too; everyone is different!
> I don't really get this attitude, it means that nothing new will ever overcome anything that's been established.
I think he's trying to quote Dan Saks' "extern c: Talking to C Programmers about C++" when he said "If you're arguing, you've lost." Saks, at least, meant that if you're striving with someone, you've already lost because your partner is already in a "frame" of mind set against your arguments that is unbreakably strong, even by logic. The talk is really good just for that aspect.
But that's a valid point.
This resonates well with me. I came from PHP and I decided to learn C in order to contribute to an existing PHP extension. I was doing well following a C tutorial. Until I arrived at string manipulation at which point I said, f* this sh.
So looked for an alternative systems programming language to learn, and found Rust. And its a pleasant language to work on. Its a low level language that feels like a high level language.
My only fear is that, if Rust becomes a popular language, the community will end up as toxic as the JS and PHP community. Right now, the Rust community is filled with the smartest and friendliest devs I know of among open source communities, and I kinda like it to stay that way.
I have yet to find a community that has gotten popular and hasn't turned into a sewage plant. I think it's just a reflection on what the general population is like... :(
Emily Dunham of Mozilla has a talk about this Community Automation: https://www.youtube.com/watch?v=dIageYT0Vgg
Name a large programming community that is pleasant to interact with for newcomers and insiders alike.
It's a level of purposeful design that only the .NET and Go ecosystems approach, and because it's unusual and largely unobtrusive, it's not obvious that this really good, highly productive community is not at all accidental.
I would say that most large programming communities split into multiple communities, so different kinds of folks don't interact as much: the people on the core developer mailing lists are mostly not the same people as on IRC, or on user mailing lists, or working on issues for 3rd party frameworks in GitHub. I can't think of a sub-community that I've had experience with that I would characterize as unpleasant.
Context is pretty important to the original comment:
> python/ruby/whatever shops are using Rust because they need a fast language, but don't want to deal with safety issues.
they're not looking for a fast language in isolation, they're looking for a fast language in the broader context of their existing technological stacks. Java doesn't help that much when you're looking into ways to speed up your Ruby application, at least not easily.
Furthermore, Java has the issues that it's only fast when memory is plentiful and it has a long startup/running time, it's "eventually fast" if you will.
Although there's JNR and JNA and it might become a standard feature at some point in the future[0]. But for a good FFI experience you also need arrays-of-structs, which means you need value types in java. So it's a long journey to get there.
Unless you need guarantees that can't be offered by garbage collection.
Edit: Check out https://dzone.com/articles/understanding-sunmiscunsafe
Hardly, compared to anything that takes user input and aims at 60/120 FPS (where you inevitably end up bypassing GC if you can't force it to run when you want to).
https://www.quora.com/Is-Java-overtaking-C++-in-the-HFT-worl...
> One thing about HFT is that the speed of the language oddly doesn't matter much. You are talking about latencies of 20 milliseconds, and any language can do 20 milliseconds since CPU times are measured in nano-seconds rather than milisecond.
and
> If you are playing in the sub 10 microsecond range doing everything in C++ helps a lot. [..] A lot of high frequency strategies run in the 20-30 microsecond time frame. I think this is where a lot of diversity in terms of setup occurs.
and
> To lower execution costs, we need a low latency system that has good determinism in response times. You cant pause for hash table rebucketing or garbage collectors or the OS taking away your time slice or moving you to a diferent processor or whatever. There are guys who try this in Java, and spend most of their time trying to make Java work deterministically. Good luck
and
> If you are in ultra HFT then C++ is the only answer (besides custom hardware - FPGA & ASIC). You just can't afford garbage collector.
(Java also has a not so great reputation about speed. That one also comes from the Java plugin, which often took minutes to start up, while keeping the Netscape browser UI locked up.)
Java is safer than C, but it's a long way behind a modern functional language like Rust.
http://www.cvedetails.com/product/19117/Oracle-JRE.html?vend...
http://www.cvedetails.com/product/1526/SUN-JRE.html?vendor_i...
The most popular(hotspot/openjdk) is written in c++/assembly
I'd love to read tutorials or books that assume Rust is someone's first programming language, period. What would that look like?
There isn't any good "first programming language" material using Rust yet, IMHO.
I'd say the greatest part about Rust is the community, which realizes all of the issues you've listed, and especially the really tough learning curve for beginners.
Generally, use the right tool for the job. I don't think I'd really want to use Rust in production yet, but it's really great for side-projects and other more experimental things.
Why not out of curiosity? I've already been showing off to people at work how much shorter Rust code is than our corresponding Java, how much easier it is to build and deploy, and how much more stable it is.
Granted I've been using it at small micro service scale at the moment, but I see no reason not to go into production at the moment (well, except for the fact that I'm the only one that will be called at night if it fails... luckily it hasn't, and I'm not worried).
(I think it's even better, but I'm biased)
One I can note is that I really like the new sections. I've never quite liked that the current book is just a bunch of chapters all in a row. Not that that's necessarily bad in and of itself, but I do think the new way is better.
!(event.ctrlKey || event.shiftKey ||
event.altKey || event.metaKey)
What do you feel is actually missing from Rust? Nothing has stopped me from replacing C entirely on the low end, and even the high end for application software development.
IDE support is easily better for java. IDEs have total type knowledge since they directly integrate with the compiler, including in long chains of type-inferred code, usually even while you're in the middle of typing and your code is in a syntactically invalid intermediate state. They also automatically manage imports where needed.
Similarly, IDE-integrated debuggers can do a edit-code, recompile, hot patch into running application while you're stepping through the code.
CPU Profilers seem about en par, but memory profilers in java are again better because they offer a combination of taking heap dumps, allocation recording and diffing dumps at the same time. While also offering good GUIs for exploring object graphs.
A solid NumPy-like library, for one.
- There's Atom with Tokamak, which you must also install racer and clippy via cargo to get rapid in-line linting.
- Then there's Visual Studio Code with RustyCode which you can also integrate with racer and clippy, that provides faster code completion and hovering over items will show a tooltip that documents that items.
- Some people like IntelliJ Rust, but I've not tried it myself.
As for installing racer and clippy, it's been made a lot easier recently via rustup:
rustup component add rust-src
rustup component add rust-docs
rustup toolchain install nightly
rustup run nightly cargo install clippy
rustup run nightly cargo install racer
Now you're good to go.
As for NumPy, there's official crates like the `num` family, which provides a plethora of useful numerics capabilities. It's not my forte though so others would know more about the best numerics crates outside of the `num` family. If you know what functionality you are looking for, it may already be created and is searchable at Crates.io.
Most Java/C# IDEs will get you:
- Type-based autocomplete (racer/YCM/etc have this for Rust)
- Jump-to-definition / documentation (YCM/etc has this)
- Autoimport
- Non-grep-based refactoring
- Auto-boilerplating: for example, type `impl Trait for MyType` and have it tab-complete to a skeleton trait impl
- Some error integration (not just fileline jump-to), with tooltips and stuff asking you how you want the IDE to auto-fix the error
- A bunch of other smaller useful things which I can't remember at the moment
Now, autocomplete is a major chunk of this, but nowhere near being all of it.
The way I see it there are two camps on this issue. There's the "text editor" camp who use vim/emacs/sublime and when they are asking for IDE support they just mean autocomplete and jump-to that works from their editor. Then there's the camp coming from Eclipse or VS who want the whole deal.
The Rust Language Server project (https://github.com/jonathandturner/rls) is working on exposing this info from the compiler in a structured form so that IDEs and "text editor"s alike can get IDE features.
Rust does have lots of numeric crates. Too many. I took a look at matrix multiply functions recently.(See [1], below "Here is what the Rust compiler actually does", for some notes on the effectiveness of Rust's subscript checking optimization.) "algebloat" wouldn't compile on stable. "matrixmultiply" is all unsafe code, with C-type raw pointers. "ndarray" has unsafe indexing. "scirust" has more raw pointer manipulation. "matrices" is an empty project.
In the "num" crate, there are bigints, along with rationals and complex numbers, but no matrices. "numeric" has matrices, but it's just a wrapper for some common C libraries. (Also, calling "panic" for a singular matrix in SVD is kind of drastic.[2])
Each of these matrix crates has its own representation for multidimensional arrays. You can't mix them easily.
Matrices in Rust really need some standardization.
[1] https://news.ycombinator.com/item?id=13230551 [2] https://github.com/numeric-rust/numeric/blob/master/src/lina...
We've had this discussion before. ndarray exposes both a safe and unsafe API for indexing, just like the stdlib vector (or core arrays). That is not problematic.
I agree that the lack of standardization is a problem, though. I think that mostly folks use ndarray or nalgebra (and num if they need bigints). Anyone can upload a crate, the question is if the crate is the main one used by the community. There are some efforts to make it easier to choose crates amongst alternatives on crates.io, however.
I suspect most of the issue here is just that no group (only individuals) is using Rust for scientific computing yet, so there's no large driving effort behind getting good libraries here. When I needed this in Rust I just picked the library that I thought would work best with very little thought, with no thoughts on writing my own or improving the lib because I didn't have time. When larger groups work on things generally attention gets paid to stuff like this and better solutions come out on top. Rust isn't quite there yet in adoption for this to happen, maybe soon :)
For 1D, though, the optimization of checking is pretty good. "get_unchecked" for Vec may be obsolete. There's an amusing Stack Overflow question [1] from someone who complains that he changed an access from "[]" to "get_unchecked()" and his program didn't get faster.
Maybe it's time to deprecate some of the legacy "unsafe" stuff. Preferably before the first CERT advisory involving a buffer overflow in a Rust program.
[1] http://stackoverflow.com/questions/39196594/why-dont-i-get-p...
Sometimes you don't want to rely on LLVM, and sometimes the size invariants aren't as easily available to LLVM.
I'm not even sure if this would be that necessary outside of the library itself; unchecked indexing would be useful to implement ops like matrix multiplication within the library, once, and probably never used after that.
The API exists mostly to mirror the stdlib one and its use cases. I don't think it's supposed to be the API you reach for usually. Just because the API exists doesn't mean it's the one you're supposed to reach for.
> That may improve, especially if there's some standard idiom for declaring multidimensional arrays and the compiler handles that idiom well.
I mean, multidimensional arrays do exist in the language. I'm not sure what you're getting at here.
These libraries provide support for treating multidimensional arrays as matrices, with the semantics of matrices (multiplication, addition, etc). You seem to be arguing for making this a first-class compiler feature? I'm not sure how that's very different from just implementing it with some unsafe code -- the bug-prone-ness of the unsafe code is about the same as the bug-prone-ness of a compiler optimization that replicates it. There is a case to be made for moving it into the stdlib itself to avoid the "10 ways to do it" issue, but Rust wants to keep the stdlib small so that's not going to happen.
This is the point of unsafe code. You use it to implement a few safe abstractions like matrix multiplication and iteration in libraries, verify that, and use them. There's a stigma attached to unsafe code; rightly so; because there be nasal demons. But we shouldn't take it to its extreme and conclude that all unsafe code is bad.
> "get_unchecked" for Vec may be obsolete.
Wouldn't count on it. A single datapoint doesn't really mean much here.
I mostly see get_unchecked being used in the context of other unsafe code. Rust's stdlib uses it for the internals of CString and for implementing iterators. It also uses it in an optimization for the rand crate; which does some trickyness with bitmask indexing that LLVM might optimize but I suspect the authors didn't want to rely on LLVM optimizing.
Servo doesn't use it at all, though. Going through my cargo cache dir the crates that use it are those which implement abstractions. Some, but not all, of the use cases there probably get optimized out anyway. The regex stuff also uses it but the invariants there are complicated enough that llvm won't optimize it (there are some decent comments listing out these invariants and explaining exactly how to use it safely, though).
Every matrix package I've seen so far in Rust turns off subscript checking with unsafe code.
[1] http://graphics.stanford.edu/courses/cs148-10-summer/algebra... [2] https://docs.rs/crate/nalgebra/0.10.1/source/src/linalg/deco...
The whole point of Rust is to put an end to that.
Fast and easy to use are UI tricks that can't be handled by a server.
Edit: I just realized that this may be read in a negative connotion and I didn't mean it like that. I just mean that I'm not the person who should be looking at those. I just know it doesn't exist yet and I'd like it to. Telling me of the Rust Server thing gives me no information as to how close the IDE is to being done.
If I look at everything I used to write in C, I'd say 80% is well suited for Go and the rest I would fallthrough to Rust for. For the stuff where having a GC and slightly less control is OK, I don't see why I would want to use Rust. Rust is just much more complex and I prefer to keep it simple stupid (KISS).
Basically, Go is good for 90% of what I used to use Java for and 80% of what I used to use C for... trying to understand where it makes sense for Rust to fit in.
(I've used Go and Rust daily for the past few years. I love them both.)
Still, the fact that all those languages are in the top 10 of programming languages kind of says that people don't really care.
Go that is?
To be even clearer: I don't think Rust's value proposition depends on whether you absolutely must avoid GC or not.
It had GC as core part of language, about six or more years ago?
PS. Pozdrawiam rodaka.
If you don't know anything about rust, you shouldn't respond to a question about rust.
The leaders of Go foster an attitude of exclusiveness (just like a month ago they wanted to get rid of the Go subreddit in favor of a solely Google owned option of Google groups).
The leaders of Rust are very receptive and helpful to new people. They are on IRC / Reddit and many other channels.
I'd much rather invest my time into a truly open language and to me, that is not Go.
To be fair, this was a proposal by a single person on the Go mailing list, and it was in reaction to Reddit's CEO publicly admitting to editing other users' comments. The person who proposed closing the Go subreddit was also under the mistaken impression that the subreddit was hosted by the Go team, which wasn't the case. In the end, there was a lot of discussion, and nothing was deleted. Tempest in a teapot, as usual.
Go has a serious culture problem, but that's not a good example of it.
In my experience, there's a lot of hostility and arrogance. More than in any other community, I've seen Go developers shut down discussion by closing comment threads on Github; reject pull requests and refuse to debate the merits of the change; castigate people for "not following proper procedure"; and being dismissive or contemptuous instead of humble when they fail to understand a problem that is being discussed (on, say, the official Go Slack channel).
My very personal hypothesis is that this is a case of mirroring. The Go development team may be said to have a laconic, authoritarian style, which is, of course, their prerogative, and which befits their position; unfortunately, a lot of Go developers seem to be under the impression that they, too, can behave like demigods. In particular, Go developers, more than other cultures, seem to get an ego boost out of telling people "no".
But this is a generalization. There are lots of friendly Go devs around, to be sure (I hope I'm one of them). I'm particularly happy with the Kubernetes team. At the same time, I do think it's an issue and one that the community needs to be aware of.
This is very misleading. There is no official Go slack channel. The slack channel you are referring is just a slack channel and is by no means official.
> In particular, Go developers, more than other cultures, seem to get an ego boost out of telling people "no".
This is misleading as well. Go was designed from the start to exclude certain features e.g. inheritance and many others. There are usually very good reasons for those decisions that people who've been following Go from its creation know and understand very well.
Now what happens when someone who does not understand those design choices comes to Go? They want their favorite feature obviously! When Go members attempt to explain them why something like that does not exist in Go (and probably will never be included) it starts feeling like "no". And people do not take "no" well. They get emotional and fail to see reason. If they had bothered to explore the language or at least read the official documentation and FAQ, that state of mind might have been avoided.
Including everyone's favorite feature does not make a language better.
Go has created a community around a certain school of programming. Nobody claims that it is better than other schools but it is a fact that it exists. But who is the one that is close-minded in this case? The new person that doesn't bother with the teachings of the school or the school that dismisses the ideas of the new person? Who is really the one that says "no"?
From my experience, the Go developers always carefully consider every new idea that is brought to the table. But it is also a fact that after the 10th time you've seen the same idea, you are not going to sit down and spend time considering it. You are just gonna link a previous discussion and say "Sorry this has been brought up before, please check these". How does that feel to a new person? I bet it feels like "no" again.
I find it pretty terrible that Kubernetes develops on Slack/Github/Videochats. For being an Open Source project, it rejects open source tooling and makes it difficult for people with poor english skills to participate (text-based meetings are much better for inclusiveness). Even if you do listen in on the video meetings, they give you a feeling as an outsider that many of the decisions have already been made elsewhere (some obscure github issue conversation, a google doc comment on a 1 year old doc, etc). It seems the only way to meaningfully participate in the k8s community is to be a known insider... :(
Wow, really? Do you have a link to that discussion? That's wild.
It should be a good summary of the event.
In Go I can only write libraries for Go programs. In Rust I can write libraries for any program.
i.e. Rust can easily produce static and dynamic libraries that are linkable with C programs and any language with a C FFI. I can write Rust code that works for programmers using C, C++, C#, D, Go, Swift, Python, PHP, Java, etc.
http://www.darkcoding.net/software/building-shared-libraries...
That's not true. It's quite simple to create a loadable shared object in go and call it using anything with a c ffi.
[1] https://www.cockroachlabs.com/blog/the-cost-and-complexity-o...
As proof, notice that it barely happens, and only as an oddity in Go, yet in rust there are actual uses (e.g. ruby and python library optimization)
I was going to say that rust has performance advantages over Go (due to GC), but look at benchmarks:
http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
Go wins some and looses some, but it's all in the ballpark (except Binary trees [1] which it loses even to Java(!)).
It's true that rust is a new language, but so is Go.
[1]: I assume it's because it's a test of GC, but Go loses to Java (which, like Go, is a GC language)
For example, many of the Java/C/C++ benchmarks are using custom optimizations that should be illegal for the benchmarking. Case in point, some are featuring custom hash maps that feature hashing algorithms that, while fast, would never be useful as they provide no protection against collisions. You'll see a hashing algorithm in a C preprocessor, for example, that just fakes having an actual algorithm whereas Rust examples are sticking to the tried and tested production-grade algorithms shipping in the standard library.
They are useful in this case, aren't they? Custom hash map implementation can be also useful in other scenarios. This is feature of the language that allow you to do that so it's not "illegal" to use it. If Rust doesn't allow you to do something while other language does, it doesn't mean it should be illegal.
I don't see any tricks in Go and it's faster than Rust and more memory efficient in almost half of the benchmarks. So what's your excuse for those cases ?
I don't speak Rust, so I can't quite tell what's going on here:
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
But the C hash function is very simple, and probably not at all collision-resistant:
#define CUSTOM_HASH_FUNCTION(key) (khint32_t)((key) ^ (key)>>7)
I am sure that the Rust could be made to do the same thing, but I am not sure that, as-is, this is an apples to apples comparison.
Another factor could also be that the GC in the Go examples just never runs since it doesn't allocate enough memory. It's hard to say exactly what's happening without tracing the runtime, which would also affect the benchmarks a bit.
It's important to note that while both languages are natively compiled (and I suspect Rust would inch out in that category due to LLVM's backing) most of the overhead would probably be from memory, whether allocations, or cache usage, which makes comparing them in microbenchmarks a little inaccurate.
This is an allocator feature independent of garbage collection; jemalloc does this for example. GCd languages have the potential to do this better since they can do additional analysis whilst tracing for little additional cost, but non-GCd languages can still benefit from this.
That is, I think I'm thinking of this:
> k-nucleotide will explicitly require built-in / library HashMap.
https://alioth.debian.org/tracker/?func=detail&group_id=1008...
Since C doesn't have a standard library hashmap, you can write an entirely custom one just for the benchmark. But since Rust has a built-in hashmap in the standard library, you cannot. Even though you could write Rust code that's the same as the custom C hashmap.
http://benchmarksgame.alioth.debian.org/u64q/knucleotide-des...
But they say 'don't write a custom hash table', not 'don't write a custom hash function'. Maybe the problem is that the data in this benchmark is just not a good way to exercise the hash tables in a given language the way the benchmark intended. That probably means the benchmark should be modified; the complaint that some implementations use 'laughably bad' hash functions that seem to be measurably decent hash functions for the data at hand seems really strange.
> Since C doesn't have a standard library hashmap, you can write an entirely custom one just for the benchmark.
NOT TRUE!
#315195 states the opposite!
"k-nucleotide will explicitly require built-in / library HashMap"
Am I allowed to use a custom HashMap for the game, or am I required to use the one in std::collections? My understanding is that it's the latter, and so that penalizes Rust or any other language that includes a map in their standard library.
In fact, the current Rust k-nucleotide DOES use a custom Hash function with a library Hash map.
No one is allowed to -- in your words -- "write an entirely custom [hashmap] just for the benchmark".
(This was all discussed to death, in early December on https://www.reddit.com/r/rust/ ).
Meanwhile, from the task description: Please don't implement your own custom "hash table" - it will not be accepted.
If you want an additional Rust hashmap implementation then you need to persuade the Rust community to provide one in std::collections.
No one is allowed to -- in your words -- "write an entirely custom [hashmap] just for the benchmark".
Again, this means that for Rust, it _has_ to be in the standard library. But C doesn't have one in the standard library. So C gets to write their own, and Rust doesn't.
I still can't get all the moral over this.
The best thing here for Rust would be to achieve the best performance on this game and nothing else.
To me this game is important and winning on it is more yet. So, if Rust don't have better result by "morals" this is so wrong that hurts. But each time I read these responses I ask myself if there is not a real problem there.
I like Rust, but performance is decisive. It is really sad to see that Rust keeps going down on this game and that makes me question myself when trying to invest time on Rust.
If, on the other hand, you're just treating the benchmarks as a fun competition akin to code golf, then by all means, hacks ahoy!
That's absurd. The benchmarks game doesn't measure real-world performance in any sense whatsoever. And many of the implementations are written in ways that you'd never ever put into production code (for example, the laughably bad hashing functions that are mentioned elsewhere in this thread). From what I understand, the Rust implementations tend to not get up to those shenanigans, which makes them appear worse than the implementations from other languages that do.
It's not the goal of a programming language to be the top on a silly benchmarks game. The goal is to be an excellent language for real-world use.
I don't know the background of these people, but they couldn't be more wrong. So that looks like "morals" or just they don't ever saw real C code out there.
For example https://github.com/haipome/fnv/blob/master/fnv.c
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
Imagine you got a million strings of variable length such that the last 4 bytes are guaranteed to be unique value between 0 and 999999. Lopping off the last four bytes is a perfectly good hash function for that kind of data.
So yeah, if you can come up with a tailored hash algorithm that works correctly and is DoS-immune for all possible input for the program, go ahead and use that tailored hash algorithm. But if your hash algorithm is only correct for the specific input data the benchmark game uses, and would not be collision-resistant under real world conditions, then you really shouldn't use that hash as it does not demonstrate anything useful.
Your argument seems to be that that's not the intent of the benchmark which may be true but it's not clear from the rules provided at all. To me, it looks like the opposite is true - they talk about using a standard hash table and most of those allow user-specified hash functions.
Btw. this site is extremly bad for benchmarks since it also measure's the startup time of the runtime in java/go/rust.
[1] https://github.com/duneroadrunner/SaferCPlusPlus-BenchmarksG...
Quote?
I've tried learning C/C++ at several times but I just don't have the inclination to have to bother about null-terminating strings, etc in 2016. I don't mind spending a little more time getting something to compile if it prevents silly mistakes.
Having said all that, it's obviously too early for me to say whether I like Rust. I'm picking it up pretty quickly since I know FP thanks to Scala, but I'll see how much time I spend fighting the borrow checker.
Feature-wise, Rust features generics and functional programming via higher-order functions and iterators, which is something that Go especially lacks in. Go doesn't have nice concepts like `Option`, `Result`, or `Iterator`. That's not something I'd personally want to live without today. The Go method is effectively writing boiler plate code everywhere, which leads to much room for error prone implementations that require more testing.
I haven't felt that Rust was more complex than Go, at least when you're actually writing software in Rust. Rust libraries feature semantic versioning and are automatically downloaded and verified at build time based on the contents of your `Cargo.toml` and `Cargo.lock` files. No importing of Git repositories directly required. Go does not provide an equal on that front.
There are a lot of great libraries out there to bring you extreme performance, simply, such as the bytecount crate, which is just a library that features a single function, a function that counts the occurrence of a specific byte, 32 bytes at a time with AVX, with additional SSE/SIMD implementations depending on what the processor supports.
All there is to truly know about Rust is the borrowing and ownership mechanism and how to implement a custom `Iterator`. If you have a solid understanding of both then you've pretty much mastered all you need to know about Rust.
The borrowing and ownership mechanism can be simplified down to: - Passing a variable by value will move ownership, dropping the original variable from memory - Passing a variable by mutable reference will keep the original variable, but allow you to modify the variable. - You may only borrow a variable mutably once at a time, and you may not immutably borrow while mutably borrowing. - You may have as many immutable borrows as you want, so long as you aren't modifying that value. - You may mutably borrow a field in a struct, and then mutably borrow a different field in the same struct simultaneously, so long as you aren't also mutably borrowing the overall struct. - You can use `Cell` and `RefCell` to allow for mutably modifying an immutable field in a struct. - You may mutably borrow multiple slices from the same array simultaneously so long as there is no overlap. - Safe memory practices means that instead of mutably borrowing the same variable in multiple places, you queue the changes to make in a separate location and apply them serially one after another.
Then for the `Iterator` trait, you would know that all traits have required methods, whereby as long as you implement the required methods for your type, you will automatically gain all of the other methods associated with the trait. For the `Iterator` type, you only need to implement the `next` method, and that looks something like so:
``` struct DataIterator<'a> { data: &'a [u8], index: usize, }
enum Token<'a> { One(&'a [u8]), Two(&'a [u8]) } impl<'a> Iterator for DataIterator<'a> { type Item = Token<'a>; fn next(&mut self) -> Option<Token<'a>> { let start = self.read; for element in self.data.iter().skip(self.read) { self.read += 1; // if next value is found then return Some(&value[start..self.read]) } None } } ```
There are only seven instances of unsafe: https://github.com/BurntSushi/ripgrep/search?q=unsafe&type=C...
Four of them are related to calling libc/kernel32 functions, which need unsafe to be called. One is due to using a memory map, which needs unsafe to be called. Only two are actual unsafe rust functions.
That's a "real-world" example though, you're asking for a more specific implementation. I can't compare because my Go would be very poor; if you posted a Go implementation, I'd be willing to give this a shot and write a Rust one.
(Mostly out of personal interest, I don't think it really proves anything larger about the two languages.)
Your benchmark proposal is interesting on the surface, but if done correctly, its speed will come down to highly optimized SIMD routines. For example, in Go, the code to search for a single byte on amd64 is written in Assembly (as it is in glibc too). This means it's probably not a good indicator of language performance overall.
Then compare it with similar tools written in Go, like sift and the platinum searcher.
The problem is, searching a file quickly is not as simple as you want to believe. If you show me a naive line by line approah, I'll show you an approach that is much faster but more complex. Top speed requires work, sometimes algorithmic work, regardless of the language you choose.
Of course, I did it myself many times. But this is NOT the point, I've already wrote it. The point is that I wrote naive approach in both languages and it's a lot faster in Go. Which is a reply to what OP wrote (don't forget where this discussion started). In this case this is the fact and I don't see any reason to fight with facts if we get bias out of the equation. In other cases? I don't know. What I would expect is that naive searching for one string in haystack to be faster than Go in a language that is performance/zero-cost abstractions oriented like Rust but its false in this case. But it doesn't mean it's false in every case. And to be honest writing that "but they have faster implementation in assembler" is not an excuse at all, you can also write your own, especially if it works so well for those languages that have custom asm for specific platforms. In the end average Joe will not care if it's hand written assembler, he will care that his naive solution using standard library without any magic is just faster.
I tried your challenge, and the first data point I uncovered contradicts this. Here is the source code of both programs: https://gist.github.com/anonymous/f01fc324ba8cccd690551caa43... --- The Rust program doesn't use unsafe, doesn't explicitly use C code, is shorter than the Go program, faster in terms of CPU time and uses less memory. I ran the following:
$ /usr/bin/time -v ./lossolo-go /tmp/OpenSubtitles2016.raw.sample.en the
$ /usr/bin/time -v ./target/release/lossolo-rust /tmp/OpenSubtitles2016.raw.sample.en the
The Go program takes ~6.5 seconds and has a maximum heap usage of 7.7 MB. The Rust program takes ~4.2 seconds and has a maximum heap usage of 6 MB. (As measured by GNU time using `time -v`.)
---
IMO, both programs reflect "naive" solutions. The point of me doing this exercise is to show just how silly this is, because now we're going to optimize these programs, but we'll limit ourselves to smallish perturbations in order to put a reasonable bound on the task.
If I run the Go program through `perf record`, the top hotspot is runtime.mallocgc. Now, I happen to know from experience that Scanner.Text is going to allocate a new string while Scanner.Bytes will not. I also happen to know that the Go standard library `bytes` package recently got a nice optimization that makes bytes.Contains as fast as strings.Contains: https://github.com/golang/go/commit/44f1854c9dc82d8dba415ef1... --- Since reading into a Go `string` doesn't actually do any UTF-8 validation, we don't lose anything by switching to using raw bytes.
Knowing this, we can tweak the Go program to great effect: https://gist.github.com/anonymous/c98dc8f6be6d414ae3e7aa6931... --- Running the same command as above, we now get a time of ~2.3 seconds and a maximum heap usage of 1.6 MB. That's impressive.
Now let's see if we can tweak Rust, which is now twice as slow as the Go program. Running perf, it looks like there's an even split between allocation, searching and UTF-8 validation, with a bit more towards searching. Like the Go program, let's attack allocation. In this case, I happen to know that the `lines` method returns an iterator that yields `String` values, which implies that it's allocating a fresh `String` for every line, just like our Go program was. Can we get rid of that? The BufReader API provides a `read_line` method, which permits the caller to control the `String` allocation. If we use that, our Rust program is tweaked to this: https://gist.github.com/anonymous/a6cf1aa51bf8e26e9dda4c50b0... --- It's not quite as symmetrical as a change as we made to the Go program, but it's pretty straight-forward IMO. Running the same command as above, we now get a time of ~3.3 seconds and a maximum heap usage of 6 MB.
OK, so we're still slower than the Go program. Looking at the profile again, the time now seems split completely between searching and UTF-8 validation. The allocation doesn't show up at all any more.
Is this where you got stuck? The next step from here isn't straight-forward because getting rid of the UTF-8 validation isn't possible to do safely while still using the String/&str search APIs. Notably, Rust's standard library doesn't provide a way to search an `&[u8]` directly using optimized substring search routines. Even if you knew your input was valid UTF-8 before hand, there's no obvious place to insert an unsafe `from_utf8_unchecked` because the BufReader itself is in control of producing the string contents. (You could do this by switching to using `BufReader.read_until` and then transmuting the result into an &str, but that would require unsafe.)
Let's take a leap. Rust's regex library has a little known feature that it can actually search the contents of an &[u8]. Rust's regex library isn't part of the standard library, but it is maintained as an official crate by the Rust project. If you know all of this, then it's possible to tweak the Rust program just a bit more to regain the speed lost by UTF-8 checking: https://gist.github.com/anonymous/bfa42d4f86e03695f3c880aace... --- Running the same command as above once again, we now get a time of ~2.1 seconds and a maximum heap usage of 6.5 MB.
In sum, we've beaten Go in CPU time, but lost the Battle for Memory and the battle for obviousness. Beating Go required noticing the `read_until` API of BufReader and knowing that 1) Rust's regexes are fast and 2) they can search &[u8] directly. It's not entirely unreasonable, but to be fair, I've done this without explicitly using any unsafe or any C code.
None of this process was rocket science. Both the Go and Rust programs were initially significantly sub-optimal because of allocation, but after some light profiling, it was possible to speed up both programs quite a bit.
---
Compared to the naive solution, some of our search tools can be a lot faster. Performing the same query on the same corpus:
ripgrep 1.13 seconds, 7.7 MB
ripgrep 1.35 seconds, mmap
GNU grep 1.73 seconds, 2.3 MB
ag 1.80 seconds, mmap
pt 6.41 seconds, mmap
sift 50.21 seconds, 16.6 MB
From here, the next step would be stop parsing lines and start searching the entire buffer directly. (I hope to make even this task very easy by moving some of the searching code inside of ripgrep to an easy to use library.)
---
In sum, your litmus test essentially comes down to these trade offs:
- Rust provides a rich API for its String/&str types, which are guaranteed to be valid UTF-8.
- Rust lacks a rich substring search API in the standard library for Vec<u8>/&[u8] types. Because of this, efficient substring search using only the standard library has an unavoidable UTF-8 validation cost in safe code.
- Go doesn't do any kind of UTF-8 checking and provides mirrored substring search APIs between its `bytes` and `strings` packages.
- The actual performance of searching in both programs probably boils down to optimized SIMD algorithms. Therefore, once you get past the ability to search each line of a file with minimal allocation, you've basically hit a wall that's probably the same in most mainstream languages.
In my opinion, these trade offs strike me as something terribly specific, and it's probably not something that is usefully generalizable. More than that, in the naive case, Rust is doing you a good service by checking that your input is valid UTF-8, which is something that Go doesn't do. I think this could go either way, but I think it's uncontroversial that guaranteeing valid UTF-8 up front like this probably eliminates a few possibly subtle bugs. (I will say that my experience with text encoding in Go has been stellar though.)
Most importantly, both languages at least have a path to writing a very fast program, which is often what most folks end up caring about at the end of the day.
Do you think you could refactor out bytestring-based string manipulation into its own library? Even better would be something that worked for all encodings (using https://github.com/servo/tendril or something)
IIRC, someone was working on making the Pattern trait work on &[u8], but I'm guessing that work is stalled.
To factor it out into a separate crate means copying the &str substring routines, since there's no way to safely use them on an &[u8] from the standard library. (bluss did that in the `twoway` crate, so we could just use that.)
It does seem like a plausible thing to do, at least until std gets better &[u8] support.
> Even better would be something that worked for all encodings
I suspect the standard practice here is something like "transcode to UTF-8 and then search the UTF-8." (This is what I hope to do with ripgrep.)
> (using https://github.com/servo/tendril or something)
I don't think I know what problems tendril is solving, so it's not clear to me what it's role is.
EDIT: and how can you have time to write this? I just usually close the browser tab when this situation occurs...
My opinion in this case is empirically checked. I am not saying this to start flame war, I am just showing my observations in particular example. I can also say that Rust regex implementation eats Go regex implementation by magnitudes (performance wise) and it will also be true and I am not looking for any flame war in this case also. I am only sharing my experience that I've backed up with proofs (code + perf results) for that particular use case. This is a factual discussion, I don't agree it's pointless.
It's 2.6 seconds for Go and 3.5 seconds for Rust, both perf results and code here: http://pastebin.com/WwhvHH6S
Your Go program is not what I would expect. A bufio.Scanner is the idiomatic (and naive) way to read lines in Go. But this is immaterial. Your results are consistent with mine. They aren't different. I just included more analysis and more programs to provide proper context to this challenge of yours.
Rust 4.6s
Go 3.1s
EDIT:
Using grep 0.6s
Then I have installed rigrep and...
Using ripgrep 0.4s
Really nice burntsushi. I am surprised by those (ripgrep) results compared to grep.
Using a `lines()` approach is naive because that allocates owned heap-allocated strings. An optimal, non-naive solution would not use heap allocation for buffering lines but use a stack-allocated array. That alone would bring significant speedup versus the line approach.
As for ripgrep, it's a pretty comprehensive application that makes use of SIMD/AVX so it's only natural that it's fast.
It took 4.6s which only confirms what I wrote on beginning when we started this discussion. Perf counters for your solution in here:
https://gist.github.com/mmstick/a8316ba0514f9d9ab33b18fa9b91...
As for timing, I'm doubtful that Go is any faster than Rust. I don't have this www.js file so I can't test it on my laptop, but I'm pretty sure you didn't even attempt to do things like enabling LTO, setting O3, disabling jemalloc, and using the musl target. All these things can make significant differences to the performance of the binary that's produced.
For me it's not naive solution. Do you have any proof it is? Can you mathematically prove that it's naive solution? I know I can't. For everyone naive solution is something different, what I saw in burntsushi reply and what I wrote myself is the closest to what I think are naive solutions.
> like enabling LTO, setting O3, disabling jemalloc, and using the musl target.
And this is for sure not part of naive solution either.
> You may not want to accept it because that contradicts your claim of the Go solution being faster
This is not true. You can find perf numbers of his solution in my second reply to him. Or you can compare those solutions yourself.
You can't prove a negative.
You are not proving anything until you post your Go code, which you don't seem to have done (please correct me if I'm wrong, I am legitimately curious to see for myself how Go and Rust stack up against each other for this problem). Until then, all you're doing is making vague claims backed up by precisely zero evidence. Why should anyone take you seriously?
When did you ask on IRC? I'll take a look.
I'm not sure what such an example would prove, other than maybe "Go is better than a subset of Rust excluding some of the core features of the language and its greater ecosystem".
This is a tricky requirement. The standard libraries of both languages use a crap ton of unsafe code. You might end up just asking for a comparison of which standard library is bigger.
But at the end of the day, both languages have perfectly standard buffered readers (https://doc.rust-lang.org/std/io/struct.BufReader.html), shouldn't a simple search like this compile to the exact same code?
Go is good, even great, at many things. But it's a language largely defined by its limitations, usually intentionally. It's an engineering language, not made for big abstractions. For me, the largest frustration is that the language gets in the way, and the pain increases with the scale of the problem. Which is to say: I think Go scales to large projects just fine, but there are problems where you'd like to build big building blocks on top of smaller blocks on top of smaller blocks, and Go doesn't lend itself to certain kinds of big, composable, data-oriented abstractions. It's small building blocks all the way.
I've bumped into several very real problems recently where Go's coarse, not-very-data-oriented imperative approach has revealed itself as a liability, and where I found myself fantasizing how I could have done it in just a few elegant lines in Haskell. Sometimes they're about expressing things simply in a composable manner, and sometimes these problems simply manifest themselves in immense blobs of boilerplate/repetition (for example, because you have to implement the same method a few dozen times on different data structures, which in a different language could be solved with a generic implementation), where Go's solution is to either eschew type safety, use slow reflection APIs, or programmatically generate the Go code as part of the build process.
Go's is also frustrating in its selective pragmatism. Where Go has chosen to automate and sugarcoat some complicated things (memory management, goroutines), it's stubbornly unpragmatic about other things (error handling, working with polymorphic data, memory safety). Go has been ridiculed for its simplistic error system, but I'm not an extremist here; I'm all for errors being values, and not a fan of exceptions. But if you look at actual Go code, a huge amount of code has to interact with errors. When nearly every function is riddled with "if err != nil", you should know that your language is crying out for just a little syntactic sugar. Or a solid type-system solution for that matter. Enums (Rust-style) and pattern matching wouldn't go against Go's grain at all, but since Go is "done", we're stuck with how it is.
I think Go's focus on simplicity is very important (I'm a big fan of the Wirth school of languages), and my worry about Rust and Swift is that they never learned this lesson. To me, both Rust and Swift looked more promising early in the design process than in their current state; Swift looks increasingly like Scala every time I visit it, whereas Rust often feels lost in a sea of punctuation. That said, my annoyance with Go is acute enough that I'm willing to deal with a few downsides if I can get a language that better matches the kinds of projects that I build.
FWIW the Rust team does consider simplicity to be important. However, it's not the only goal, which means that sometimes it has to be sacrificed to be able to make something possible. But most new language features do get discussed in the context of a "complexity budget".
So Rust tries pretty hard to ensure it doesn't get more complex than it has to; but it doesn't put simplicity as the overriding goal and hang all else to get it.
> Enums and pattern matching wouldn't go against Go's grain at all
I've said this many times before -- I don't miss generics from Go; I understand why they're not in the language. I miss ADTs and pattern matching.
This is a must for me. I often give up on compiled languages because libfoobar isn't yet available on the language and I have no interest in writing (and maintaining) language bindings for third party libraries I use.
My dream is that one day I'll be able to do something like:
Foobar = link_cimport({"headers": ["foobar.h"], "packages": [ "foobar" ], "prefix": "foobar_"});
Foobar.do_something(); // would call foobar_do_something();
Foobar.hidden_function(); // would call hidden_function();
Foobar = loadable_cimport({"headers": ["foobar.h"]});
foo = Foobar.load("/path/to/foo.so");
bar = Foobar.load("/path/to/bar.so");
It converts C headers to a Rust module containing the relevant type/function/etc. definitions. On stable you need to either pregenerate the module (not too bad if you're pinning particular versions of libraries anyway) or add a build script to autogenerate it. On nightly, there's also a compiler plugin that does all the work for you, and looks not-too-dissimilar to what you wanted. The only thing it lacks is the prefix removal stuff.
Blog posts are one way to showcase tooling, perhaps not this one, but they have their place.
Which is undersold in the propaganda! Safety, ehhh I'll take it but I'm not jonesing for it. A package manager that works and has adoption? Thank you! I've wasted far too much time wrangling C++ dependencies by hand...
It can do (roughly) everything C can, i.e. it's natively compiled, gives control of memory layout, and doesn't depend on a GC runtime.
Previous C killers were either dependent on a fat runtime (which isn't a big problem in general, but is a problem for some of niches dominated by C), or didn't offer meaningful improvement in safety/concurrency/expressiveness.
* "First viable replacement for C" - C++ was invented decades before Rust.
* "Well-written C programs translate almost 1:1 to Rust" - C programs can be compiled with minimal changes as C++.
* "or didn't offer meaningful improvement in safety/concurrency/expressiveness." - C++ offers major improvements in all of those areas.
A lot of C programmers don't want to switch to C++ and they won't want to switch to Rust either, because they favor simplicity and neither Rust nor C++ are simple. They'll probably switch to golang for some stuff if anything and just stick to C for the rest.
Rust is complex because it's powerful.
C++ is complex because it's a mashup of overlapping features from different eras.
I don't write C++, because whatever I write will crash for subtle reasons and I'll be told it's my fault, because of course you don't do this when you use that feature.
I can write Rust with all its complexity, and as long as I get it to compile, I'm confident it works more reliably than any C++ that I could have written.
Note: Rust executables can be trimmed down by stripping them (not done by default, even for release builds) and using libc malloc instead of jemalloc. But even then, C wins by a lot.
Relevant blog post: https://lifthrasiir.github.io/rustlog/why-is-a-rust-executab...
C links dynamically to its libraries. Rust does not (by default).
This overhead is a constant overhead so it rarely matters. When it does, you can strip it down further and get it to the same level as C.
This is not evidence of a fat runtime, just a different default compilation strategy that prodces larger binary executables. jemalloc is (an optional) part of the runtime, but the rest isn't.
I suspect rust hello world using those libraries is due to a lack of LTO? Hello world doesn't need any of those.
You'll get different results if you compile them with musl for truly static and fair binary size comparisons.
if I remember correctly you should compile with --nostd flag and provide _start function for loader to load your executable!
you can do same with rust too (But I think you should provide a special flag for compiler). You can find all of it on documentation.
I've written C-like programs in Rust, and that goes very well. But I really like function overloading, generic operators that I can overload from the left and the right, integer parameters for my templates/generics, copy semantics as the default (opt-in for moves), placement new, explicit destructors, and maybe a few other niceties. Rust does none of these things the way I want, and so I'm stuck with C++.
There are a few other things where I think Rust just made the wrong choices and didn't improve over C++. I've always hated the dichotomy in C++ strings between const char* pointers and an actual string class. Rust had the chance to make strings feel as comfortable as integers, but instead they introduced their own dichotomy with String and &str.
Speaking of integers, Rust's choice of 32 bit integers as the default for literals is painful given that every machine most of us will ever care about is now 64 bit. I routinely deal with arrays and files that are too large for a 32 bit integer. This means I have to remember to suffix all of my integers in for loops etc... C++ has this wrong too (backwards compatibility), but Rust had a chance to make a clean start and did not.
It makes perfect sense when you understand the differences and reasoning behind it. A `String` is a heap-allocated string that can grow in size. On the other hand, a `str` is basically a fixed-size string array, but you'll never interact directly with this type because there's no point in it. It's tucked away inside the `String` that created it.
Meanwhile, an `&str` is a slice of that fixed-size `str` array that's hidden within the `String`. You can think of a `str` as an `[char]` with it's own special `str` methods, and an `&str` as an `&[char]` which has access to all of the `str` methods as well.
> Speaking of integers, Rust's choice of 32 bit integers as the default for literals is painful given that every machine most of us will ever care about is now 64 bit. I routinely deal with arrays and files that are too large for a 32 bit integer.
I'm pretty sure the default integer is a `usize`, which is 32-bit on 32-bit systems and 64-bit on 64-bit systems.
Rust will type error if you try to use u32s with an array so it's all good though. Just means that you need to specifically `: usize` things.
If you need an integer type for counting stuff, u32 is fine. If you actually need to talk about memory, use usize. The compiler will force you to do it.
Maybe I'm missing something, but it seems to be i32: https://play.rust-lang.org/?gist=c79d4afef7fa20c81ba14de61f5...
Arrays are indexed by usize, so if you're on a 64-bit machine, then you shouldn't need a cast. It's _unconstrained_ numbers that default to i32, not anything without a suffix.
I would prefer a str be a str be a str, regardless of how you got it. Lowercase type-name and fundamental like an integer.
I'm fairly certain I understand why Rust made the choice they did. I've read the forum threads at HN, Reddit, and users.rust-lang, and I've seen previous replies by you and other Rusties, so I hope you won't try to educate me about the performance advantages of having slices as references and another string type as an ownership class or why we need OsStr and friends.
If strings are the fundamental processing concern in your application, then I think you should be able to opt-in to that kind of micro-optimization and complexity, but it would've been better to spare the rest of us who have different concerns. I don't want to become a string expert to build a filename, and the default implementation could (at least conceptually) be always on the heap for all I care. Go one step further and implement the "small string optimization" (Alexandrescu's fbstring), and you'd probably get back most of the performance without nearly the complexity.
> Arrays are indexed by usize
I've spent the last half hour trying to troubleshoot why this line hangs:
let aa = vec![1u8; 10e9 as usize];
Anyways, I wanted to test the commented out line, but I'm stuck for now. If that line would work, I'll admit I was wrong, but I think the uncommented line shows a similar complaint.
for ss in 0..63 {
//print!("{}\n", aa[1<<ss]);
print!("{}\n", 1<<ss);
}
Looking at the present and the future, why is that a sensible default? Both x64 and ARM are going to use a 64 bit integer register for the operations, and many of those operations are going to be 1-clock throughput. You can probably find a counter example, but 32 bit integers aren't generally faster than 64 bit ones.
The lowercase type names are reserved to primitives. The String type is not a primitive but a comprehensive data structure, hence the capital S. The String type contains an `str` primitive though, along with size information.
> I don't want to become a string expert to build a filename, and the default implementation could (at least conceptually) be always on the heap for all I care.
Is it that hard to understand that when you create a string, you will create it as either a `String` or `PathBuf`? File methods are designed to automatically convert input parameters into a `&Path` so it doesn't matter what string structure you provide.
There is also no way (currently) to create a stack-allocated string with the standard library out of the box. You can do this with crates like `arrayvec` though. It's very much opt-in for that performance.
let path = String::from("/tmp/file");
let mut file = File::open(&path).unwrap();
> Looking at the present and the future, why is that a sensible default? Both x64 and ARM are going to use a 64 bit integer register for the operations, and many of those operations are going to be 1-clock throughput. You can probably find a counter example, but 32 bit integers aren't generally faster than 64 bit ones.
No need to use a 64-bit integer when you only need a 32-bit integer. You can fit two 32-bit integers into a single 64-bit integer and perform a calculation on both simultaneously with a single cycle, versus spending two cycles to calculate two 64-bit integers. There's also no need to pay that memory cost either.
No, I'm interested in what tradeoffs you would have made differently. I now understand. Thanks! (I disagree, but at least I understand.)
> why this line hangs:
It compiles and runs effectively instantaneously for me on Ubuntu under Windows as well, so that's very strange. Maybe file a bug?
> I think the uncommented line shows a similar complaint.
Yes, there's no constraint on that literal, so it's going to be an i32. When we made this decision, we did some analysis, basically no numbers in real world programs weren't constrained, it was often tests, toy programs, and documentation. It should be a rare thing. YMMV.
> why is that a sensible default?
Your assertion about the speed was the opposite of what was asserted while we had the discussion, basically. And not everybody is running on 64-bit hardware, so it's a broader default.
Follow-up: I tried it with -O (don't know why I didn't think of that earlier), and it runs fine. So maybe the debug version is just generating terrible code, initializing by iterating through 10 billion bounds checks or something?
Anyways, more importantly, it works as I would like and does not behave as a 32 bit integer. I think I understand what you mean by "constrained" now. And clearly, I was wrong.
However, if most un-suffixed integers in real-world programs will become constrained (as you claimed), this further confuses me why i32 is the unconstrained choice. It doesn't seem like something so rare could be enough of a performance problem to justify being anything but the largest supported size.
I remember installing it by cut and pasting one of the "curl ... | sh" commands there.
> Nothing about that code should be doing bounds checks, as it's just allocating an array.
I didn't dive into the macro definition for vec!, but I assume there is a loop in there to Copy the initialization element 10 billion times. I think you guys do bounds checking on the lower level reference to a slice that Vec uses. But I really don't know. If it's not that, then it was hanging or spinning doing something else. (Debug version of your memory allocator?)
The fact that C and C++ compilers generally chose to leave int at 32 bit on 64 bit platforms, combined with the standards requiring "usual promotions" for smaller types to go to int bites me all the time. I'm very happy that Rust dodges the promotions problem altogether, and I'm sorry if I'm wrong about the array subscripting thing (does the snippet I provided panic at 1<<32 or 1<<34?).
> And not everybody is running on 64-bit hardware, so it's a broader default.
That argument could be used to justify 8 or 16 bit integers... :-)
Overflow is a "program error", and in debug builds, is required to panic. In other builds, if it does not panic, it's required to two's compliment overflow. Rustc currently just overflows, but in the future, we'll see.
That's true except our 16 bit support is nonexistant at the moment :)
If you pay attention to your usage statistics, I'll bet you drop 32 bit support before the May 2025 deadline we were discussing in the other thread.
I do too, and I'd like to think that rust needs all those things to be a replacement, but realistically I think that the only killer feature that rust is still missing is reasonable interoperability with C++ (which admittedly might require implementing a few of those features).
Disclaimer: I have been following rust since Graydon initial announcement, but I have yet to write a single line of code in it.
If you're stuck with C++, might as well do it safely:
(shameless plug) https://github.com/duneroadrunner/SaferCPlusPlus
I might be wrong though. What happens with your vector in this code?
using namespace mse::mstd;
vector<double> data(10);
double& dangling = data[0];
data.resize(100000);
double crashing = dangling;
The problem here is there is only one operator[] for both reading and writing. This is a simple contrived example, and taking a reference like that looks artificial, but there are a lot of other ways in real programs to stumble on to this. (I don't think it's as bad as the Rusties do, but I stumble into this bug once or twice a year...)
The Rust folks seem to believe you need a borrow checker to solve this problem, but I think that a different container library in C++ could do the trick. For instance, favoring value copies instead of references, and returning a proxy object from operator[] instead of a reference.
double& dangling = data[0];
auto not_dangling_iter = data.begin();
// not_dangling_iter += 0;
And while I can overload the & (address of) operator to "prevent" you from getting a native pointer to a "safe" object, I don't know if there's a way to prevent you from getting a native reference. If you wanted to somehow enforce a prohibition on the use of unsafe C++ elements (like references), that would probably require some sort of static tool that is not yet available. But should be fairly straightforward to implement, I think.
But if you just want some confidence in the safety of the code you write, it doesn't take much effort to reliably avoid using C++'s unsafe elements.
Oh yeah, maybe. But that would still require the ability to overload the dot operator, wouldn't it? And how would you know when to deallocate the proxy object? And presumably there would be some run-time overhead. Hmm, I don't know if it wouldn't be more practical to create a static tool (or "precompiler") to automatically convert references to (safe) pointers (or iterators).
mse::mstd::vector<mse::TRegisteredObj<std::string>> data(10);
data[0] = "some text";
mse::TRegisteredRefWrapper<std::string> dangling = data[0];
data.resize(100000);
try {
std::string crashing = dangling;
}
catch (...) {
// expected exception (not a segfault)
int q = 3;
}
[1] https://github.com/duneroadrunner/SaferCPlusPlus#tregistered...
Yes, the language is supposedly not undergoing breaking changes without strict deprecation and feature-gating. However, as new features are added to the language, libraries are usually updated to leverage them, and suddenly you are also forced to keep up with that rapid release cadence to ensure your project still builds.
Meanwhile, C code from the mid-80s still compiles with modern compilers, often without any changes whatsoever, and without any feature gates.
Don't get me wrong. I like Rust and what the project is trying to do. But I see two major problems that are preventing me from using it for anything major at the moment: 1) the extremely rapid rate of change, and 2) difficulty of implementation (both of which, taken together, lead me to expect no serious alternative implementations any time soon).
Before I will be willing to adopt Rust as a C replacement in full, there needs to be a solid, stable language definition that compiler writers can target and programmers can rely on for a good chunk of time. I have a feeling we'll get there eventually once the number of new, useful features that are missing from the language starts to dwindle and it begins to stabilize naturally.
In the meantime, I'd settle for an "LTS" release or something. That could work.
Your dependencies don't upgrade themselves. Its true that if you upgrade your dependencies, they may depend on a more recent verison of rustc, in which case you need to upgrade rustc as a part of upgrading that dependency. But if you are using rustup this is not harder than upgrading those dependencies - probably easier in fact.
In exchange for downloading a tarball at most every six weeks, you get a language that is developing new features.
The actual Rust API has changed very little since the 1.0 release. All the changes that have occurred have been mere additions to the language -- nothing breaking. Hence why the 1.0 release was termed as a 1.0 release. There won't be anything happening that causes breakage until a 2.0 release.
You'll find that C compiler and library development is also rapid like Rust. There's new versions of LLVM/Clang and GCC being released on a regular basis. There are constant updates to musl and glibc. These changes tend to break far more often than Rust, which is developed in Rust and can be guaranteed to be much safer.
That Rust is being openly developed on GitHub with a significant number of developers, with an official compiler + library + doc + comprehensive suite of resources collectively being maintained, that's a significant incentive to work with Rust. There's a solid RFC process to prevent anything silly from entering the language. No random decisions by random people -- it takes a lot of convincing to add a new feature to Rust.
Anyway, I don't think that's a problem with my concept at all. What I want for Rust, and what I think will eventually be feasible for Rust as the language matures, is a standard that can be shared by compiler writers and Rust programmers. A standard is worthless if it's inadequate (see ISO Pascal or ISO Basic, for example), so it's a good thing that right now Rust is iterating quickly and improving rapidly. But eventually, there's going to need to be a stationary target for compiler writers to aim for, and for Rust programmers to be confident that the code they write today won't be littered with deprecated features, anti-patterns, and worst practices tomorrow.
Frankly, I don't see wide adoption happening in the corporate world until there's an LTS compiler at the very least.
What you're describing is the spec and not the implementation. An implementation would be glibc (GNU Libc with open source proprietary extensions) and musl (MIT implementation that strictly adheres to spec). These libraries are updated often.
Meanwhile, Rust offers more than just a specification. It offers an implementation. This is libcore-rust + libstd-rust. This is what receives updates each 6 weeks. It also offers a compiler that comes with it, rustc, which is also what receives updates each 6 weeks.
Rust strictly adheres to semantic versioning, so changes to the API are not allowed without a major version bump. Code that was written for Rust 1.0 will still compile on Rust 1.15. Code written for Rust 1.15 may not compile for a Rust 1.0 compiler though because bumping the second version implies adding new features. Rust doesn't yet utilize a patch version though. There's never been any critical issues discovered in each stable release.
> Frankly, I don't see wide adoption happening in the corporate world until there's an LTS compiler at the very least.
Can't say that I understand why a LTS compiler would be required for adoption. Many in the corporate world are already using Rust, either publicly as an official friend of Rust or privately in smaller projects. There's nothing a LTS compiler would provide that would be beneficial to the corporate world. Semantic versioning and the ease of managing Rust toolchains with rustup has everything pretty well covered.
As to my own concerns (rather than corporate concerns), here's where I'm coming from:
When I wind up choosing C for a particular project, I don't use it because I like it (I don't particularly like C). Typically, I use it because there are three factors I've considered: 1) This particular project needs the low-level access or performance characteristics provided by a systems language, and 2) C is such a language, 3) C is ubiquitous.[1]
The thing is, every platform for the last 25 years or so has had a reliable ANSI C compiler. If the goal of my project is to be cross-platform, then a carefully-written application in standard C is guaranteed to be able to reach the widest number of users, many of whom won't have to do or install anything to build my application, since many environments already include a C compiler without any additional installation necessary (think Unix, BSD, a shrinking number of Linux distros, ...). Using C puts my application on the path of least resistance for potential users.
Similarly, if I'm writing a library, every language with an FFI can trivially access a library written in C. Even without an FFI, most languages can interface with C code through some kind of bindings (like the Lua-C API, for example). Again, choosing C is a path to wide adoption.
Again, I don't particularly like C as a language. However, right now, it has an undeniable advantage as a platform. I do like Rust, however, and I would love to see Rust give C the boot. In order to do that, it needs to eventually (one day when the rate of additions/changes to the language has slowed and it has grown more stable/mature) offer a platform that can be relied on to "just work" on any given platform with minimal fuss and without any sudden changes. As such, both the "end-programmer" and the compiler-writer in me would like to see some version of Rust (again, eventually) that is "set in stone" and widely available, which really would solidify that notion of stability/maturity.
I'm not advocating for Rust to stop innovating. For all I care, the "standard" can be a simple snapshot of some state of the language, the way the Haskell Reports seem to be nowadays: GHC keeps on iterating and innovating the language, yet there's still a standard updated every x years that provides a static target. But for that designated snapshot, I want to see a compiler (or two, or several) and a bunch of libraries that are supported for the long haul.
It's simply a matter of "C has this, it's an important property thereof, and any replacement will need it, too".
[1]: Some other projects demand C because they constitute contributions or extensions of other projects that are already written in C; e.g., kernel code for an existing OS that is written in C.
The fact that someone says on HN "we're backwards compatible" doesn't quite offer the same level of trust as an ISO standard. Especially when there are some conflicting reports out there i.e. there are many super-awesome libraries that only work with nightly (I think it's called), because many of those maintainers are early adopters that love to try out new features.
Programming languages and libraries are building blocks, I don't want them to change under me in the middle of a project. This is why I dropped Swift, the 1.0 -> 2.0 migration was a very painful, and from what I hear recent major version changes aren't running smooth either.
This is totally orthogonal to whether or not Rust stable is backward compatible, which it is. People put a lot of work into making sure it is, and this sort of FUD is very frustrating to read.
Then you shouldn't take the word of a random person but from the Semantic Versioning guidelines. If you're not following the semantic versioning guidelines then you're doing it wrong.
> Especially when there are some conflicting reports out there i.e. there are many super-awesome libraries that only work with nightly (I think it's called), because many of those maintainers are early adopters that love to try out new features.
Totally irrelevant. Very few libraries, if any, require a nightly compiler. The libraries that do offer features that require a nightly compiler have those features as optional. Anyway, once the new Macros update releases, the need for that nightly compiler will go away. These libraries are just taking advantage of compiler plugins.
> In the meantime, I'd settle for an "LTS" release or something. That could work.
This is something we've discussed. I imagine it will happen. At first, I would expect six month LTSes, then longer ones as time goes on.
The company that employs me just upgraded to a C++11-capable version of GCC three months ago. Even C++'s three year cadence seems like it might be too fast! Thing is, the company wants to vet our entire platform, which itself only changes yearly. They want to know they can get bug fixes, and vetting/upgrading the compiler is a lot of work regardless, so they tend to do one and stick with it for a while. OpenBSD does the same thing for security reasons. If Rust wants to replace C and/or C++, these are things that will need consideration eventually. I don't see my company allowing Rust any time soon, unfortunately.
As a compiler-writer, I'm more concerned that Rust is a quickly-moving target at the moment. I think that as Rust becomes more mature, it will naturally begin to stabilize feature-wise, in which case something akin to the Haskell model would work well, with innovation happening in the flagship compiler and "snapshots" of the language being taken as standards periodically. That's just a matter of time, though.
And why should one invest time for such a low-level component in their toolkit every 6 months?! I want to build software and solve problems, not spend my time keeping up with programming languages and library updates.
You seem to come from an experience where software updates are kinda sorta mostly backwards compatible. Rust works very hard to actually be backwards compatible. There is infrastructure to check for regressions across the entire package ecosystem, which is run regularly on HEAD and on any commit we think might potentially be breaking before we even land it.
You say that like it's a good thing that with C, you can ignore 30 years of progress in software development. Colour me unconvinced.
That's not a responsible metric. 30 year old software was written against 30 year old APIs vulnerable to known attack vectors, and with 30 year old notions about security. The belief that this software is suitable for modern use is absurd IMO.
And if the types of programs you're talking about are small UNIX-like utilities, well then they're small and easily updated aren't they?
Lots of software we use day-to-day is old. For example, a couple years ago I fixed a 20-year-old bug in netcat, which itself hadn't been updated since the mid 90s. After I fixed the bug, I didn't need to fish out an old compiler or any compatibility libraries; I just compiled it with the latest GCC and glibc, and carried on with my day.
Don't get me wrong, I like Rust a lot and I'm excited that there's finally a real contender for the systems programming throne after all these years. I'm just less enthused about the prospect of refactoring my codebase once every few months to stay current. This is something that I think will level out with time, though. C itself was almost 20 years old before it was standardized, after all. I don't think it will take Rust that long, and I look forward to the day my copy of the Rust standard arrives on my doorstep.
Let's assume this is true, brand-spanking new software will be written against 30 year old APIs that have survived 30 years of attacks and audits, but will avoid the ones that have proven vulnerable.
> I'm just less enthused about the prospect of refactoring my codebase once every few months to stay current.
The perception that you need to do this is false. You might feel pressured to refactor simply because of new syntactic improvements that make code clearer (like switching try!() macro to infix ?), but that pressure is a fiction. There's no real need to do it.
Could you show me an example where post-1.0 you wouldn't be able to do this with Rust?
Except most people don't consider those as criteria at all. The only criteria considered are "does it run and produce the output I want".
What the rapid releases do affect is the rate at which new releases of libraries stop working on older compilers, which absolutely has to do with ignoring progress.
Or stop using other peoples' code and just use core language features and build everything from scratch. Which is what I end up doing, and it has some benefits. But it definitely slows you down.
What are they going to teach in school? What are new systems programmers going to start with? I suspect it'll be rust and go, not c.
Or is it?
I regularly use six different languages of three paradigms and various abstraction levels, and this doesn't count DSLs nor Turing-complete tools like make. Of my colleagues from current job, half of the more experienced set write in at least three languages as well. Of the team from my previous job, all of them are polyglots. And of course none of us started with all the languages, we picked them up as needed and wanted.
To recap this has been my life as a programmer: Java -> JavaScript -> PHP -> Python/C/Assembly
This dream that most people have made up about programmers not switching languages is a dream. I switch whenever one language can do a task better then another. I don't get caught up in "best language" fights.
I play with a lot of languages but I've invested a lot of time into learning python and getting a nice python workflow going. It would take a lot for me to switch to another primary first language.
I work at a tech company that's been around since the 90s and the guys that were writing perl and java in the 90s are still writing perl and java. You'd have to drag them kicking and screaming into using a different language.
What may look like "bah humbug I'll use X because I like it" may just be picking a language on concerns not limited to features. Looking at social concerns is important as well. Who is maintaining your code after you're gone? Company tooling for certain languages? Will I get fired for using it (AI winter)?
Lots of big questions
Do you have any data to support this claim?
It crosses boundaries.
Blog posts wont pull me away from C, tooling and docs will.