Rewrites of services to decomp into different services, migrate to different datastores to scale up etc. is completely justified and is a good problem to have.
Compare Go and C++, and I bet C++ has had more language changes since Go's 1.0 than Go has. That makes large portions of C++ "relatively new and immature" compared to Go.
Furthermore, A lot of companies to this date have there performance critical code written in C++ , C or Java
Such as you can take HFT for example they use C++ for most of their trading applications instead of newer languages
I'm definitely not saying C++,C or Java is better than Rust or Golang each language has its own pros and cons
High performance systems in GC languages often preallocate or use object pools to reduce time spent in the garbage collector. LMAX's Disruptor, for example, allocates all memory at startup; no additional memory is allocated so the garbage collector never runs. Generational GCs make this strategy more effective because long-lived or large objects are traced rarely or not at all. Another option for trading systems is to run garbage collection after hours when the market has closed.
Go's GC is conservative, not generational, and has a hard coded collection interval of 2 minutes; you pay for a GC even if no memory has been allocated. There are only a few configuration options for Go's GC; if it doesn't work for your use case then you can either fork or use a different language.
This is more about scale and saving computing costs, than proving out a business idea or implementing new features.
One of the big issues with Go is that it's got a steep learning curve. (There was an article about this posted on HN yesterday, in fact)
Rust, by contrast, is an extremely easy language to learn. Of all the languages I've picked up at least partially in my ~12 years of professional programming, Rust was probably second only to to PHP in "get up and start running with it".
If Discord is trying to pick up new developers, or cross discipline seniors to improve their talent pool, Rust is definitely a better choice than Go.
I wish we had a good way to evaluate this question in a quantitative way, but I haven't seen one yet. So reasonable people are going to disagree on the answers here.
I am sure rust is great but it is at the very least more difficult to learn then go.
I don't get Rust. I got Go very easily. Rust just doesn't make sense for me.
In Go I can just write down my thoughts and they translate to code easy. I don't even know which mental game I have to play in Rust. And all the conventions in Rust are another step. The final nail is the un-community of Rust. I still have no idea how to write a simple RESTful http server with a mongodb backend. I've asked only to have been met with downvotes on Reddit. The official MongoDB discourse didn't answer either.
Time and again I want to do something with it but every time I just quit frustrated and irritated of the complexity of everything in Rust land.
Think of Result as (foo, err) bundled into one thing. result.unwrap() is then equivalent to:
if err != nil {
panic(err)
}
if err != nil {
return ..., err
}
Don't get me wrong, there are complicated things in Rust, I just don't see how Result-based error handling is one of them. This is also not a new idea, many languages have something similar.
Also mature languages, even though they're more stable, are just more of a pain to work with since they don't have a lot of the new shortcuts.
For something like Discord, Rust is perfect.
To get actual product work done with optimal efficiency, a mature ecosystem will always win.
Maintaining a codebase written in C is far far harder than one written in Java and that's far harder than one written in Go.
Personally i think developer skill has far more to do with maintainability than language choice. And I have worked with a lot of shit code.
Just like how they did in this case completely switching from Go to Rust
I'm sure we're glossing over probably hundreds of hours of investigations by dozens of people.
The service itself is pretty simple, so it wasn't a gargantuan task. So it was either, rewrite this again in Go (using off-heap data structures to avoid GC pauses, which makes the software both harder to maintain, and also less idiomatic) or try a new language where we won't have these issues - where we don't have to resort to clever hacks to work around the runtime system.
Since then we've continued our investment in Rust, after success of this project (and many others we've yet to blog about as well.)
It seems like everyone is just applying their subjective view. Which is fine. Just don't assume your equation works for everyone and every business.
C++, C and Java are old, so some mistakes can be forgiven. However there are problems with these languages that new languages have addressed.
Do you think a car from 1985 is better than one from 2015? The same applies to programming languages. Given the explosion of programming, it is absurd that the best ideas from PLT are not in mainstream languages.
A complete re-write (preceded by a very deep analysis) is exactly the opposite of productivity.
> problems with these languages that new languages have addressed.
And new problems were introduced... case in point: this very article.
> Do you think a car from 1985 is better than one from 2015?
Depends on the car...
I think this analogy would be better: "Do you think a hammer and screwdriver from 1985 are better than the ones from 2015?"
Plus most of the organizations still write code in matured languages such as C,C++ and Java.
As others have said, however, a big factor certainly is that an internal lead developer lobbied for Rust, which they experimented with.
Scala has already found its niche in Big Data, streaming and distributed systems. Usually, the stack that you can find in my area, consist mostly of Java codebases, sprinkled with some Scala services that are purely data-oriented. There's a high probability that Discord is already using Scala.
I would say it's cool _because_ of the technical reasons, though! You make it seem like people are inventing technical reasons to make rust seem cool. I could make a long "technical reasons" list why I would personally prefer to work in Rust, but it boils down that it makes me more productive, and the code more performant with fewer gotchas and less sacrifices/compromises (compared to say C++ or C). Do I think Rust is cool? I guess... But that which attracted me to Rust was there already; I didn't have to make it up[1].
I understand not every language is for everyone. I've had colleagues dismiss rust simply because they prefer C++ (and we work with C++). Some will even agree Rust is better when presented with technical reasons, but still not like Rust "just because."
> I wonder why they didn’t try doing things like putting the LRU cache into foreign, unmanaged memory to avoid garbage collecting it. Easier said than done, but why not?
I write Go intermittently since I participate a little in the Kubernetes ecosystem, and haven't dwelled that deeply into it. How would this be done without reaching for C, though? Or would it be fitting to swap languages in this "subset" of the implementation?
[1]: I feel like "make up" is the wrong phrase here, but I can't think of a better one right now. I guess what I want to say that Rust's "cool" reputation is well deserved. I don't think the technical reasons are made-up to serve as office-politics cannon fodder.
It reads like a tool (Rust) looking for a problem.
now its php->rust so a widely different migration but rust was measured for its latency cutting value, not just fad factor
I'd believe discord and others are on rust for this and not just the new shiney (that would be to their loss)
The most recent example was a replacement for a complex YAML DSL, served by spring config sever and interpreted by a python program on every instantiation. This thing was literally using 20% of our CPU cycles over the whole compute cluster and estimated to cost around €6k per month. Long story short I provided a fully working pre-processing solution with effectively no runtime cost, no dependency on the spring config server and a more powerful & type-safe DSL. It was dhall with a bit of bash and python, although you could also have used jsonnet or any configuration language. Another developer with more clout pushed me out in favour of their own solution, which I believe was to keep me away from his political pedestal. A few months later I left and from what I've heard the company is still burning 20% of it's compute budget on this...
So far though rust-analyzer >>>>> JetBrains' plugin
Ah yes, because what I really want out of an ostensibly memory-safe language like go is to have to resort to C, or what amounts to C, to get anything done. Or I could use a language which doesn't have this garbage issue. But that would just be political nonsense apparently.
The article is about a very specific service handling state for if you've seen a message in a channel. It has nothing to do with processing user content.
"Principled policy and software design" is exactly what's needed to get rid of memory bugs in C too! So we can conclude that these things are impossible, or else we would not need memory safe languages in the first place. Since it's increasingly apparent that go requires these mythical safeguards after all, it's safe to say that it is not a memory safe language in the useful sense.
And here the cache is essentially the entirety of the micro-service:
> [The service's] sole purpose is to keep track of which channels and messages you have read.
so it's a giant map of (user, object) -> read flag or something along those lines. The DB commit and service input would be very light, and not really your main concern.
We hit the performance ceiling of redis, and rather than deciding to go with redis-cluster, and dealing with the headache of managing persistent redis, we went this way, so we could consolidate data storage to scylladb/cassandra (it was cassandra at the time of rewrite, scylla now since we've migrated our data stores over).
We have a bunch of operational experience with C*/ScyllaDB.
Such a theory is more robust if it predicts how the organization will work in general. Thus one would also be partial to suspecting that neo-languages are declined for political reasons as well -- that a few powerful voices simply prefer to find other solutions, and then find post-hoc technical arguments.
Such a theory works less well if it is only used to explain some particular (ostensibly) technical decisions.
Every time I see a company post about how they’re changing programming languages, I tend to believe it actually happened because engineers with enough political clout decided they wanted everybody to change to their preferred language, not truly because of technical reasons, but because they just wanted to be paid to write in a cool language. But, of course, you’d be lambasted by your colleagues if you didn’t provide technical reasons, so you go and hunt for technical reasons that seem insurmountable and build a case for it, turning a blind eye to perhaps less exciting solutions that don’t require a programming language change.
(It’s sort of like how a cop in America can find you doing something illegal in any circumstance if they look hard enough. Except in this case, it’s a matter of finding technical deficiencies.)
This strategy highly depends on clout though, which is why I think technical arguments are frequently really a façade. If, say, a junior programmer doesn’t have clout, even presenting an equivalent technical argument won’t lead to a change in programming language, especially if it’s the senior’s favorite. I’m sorry to say, but most tech organizations aren’t meritocratic or egalitarian in this way (and perhaps they shouldn’t be).
As it pertains to the article—which again I don’t claim is all “fake” politicking (I don’t know anything about or anyone at Discord)—they mostly ripped on garbage collection. They specifically talked about how GC was bad because their LRU cache had to be scanned, which was part of why they observed spikes in their metrics. I wonder why they didn’t try doing things like putting the LRU cache into foreign, unmanaged memory to avoid garbage collecting it. Easier said than done, but why not?
I think using any language with any serious and large project will find deficiencies in the language. At some point in a product lifecycle it’s more profitable to just work with and ameliorate those deficiencies than do rewrites where the new deficiencies will be completely unknown.