So it's just like C++ ;) except it has pattern matching and const by default.
…it’s:
> And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches.
You know, those hugging face guys kindaaaaa do know what they’re talking about.
Like, really.
Python is a pain in the ass in production environments.
I really dislike blanket statements like that one. Because any programming language can be a pain in production environments. One of the disadvantages of Python is also one of it's advantages. Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points is so mature. With those in place Python becomes great in production.
Yes it'll require your organization to take some serious CI culture decisions and also enforce them. The thing is, however, while you can get a long way without doing that with some programming languages, you're frankly always going to want them by the time you reach enterprise size. So it's just so much easier if you get it done right away, and you will, if you're serious about using Python.
I'd argue that if it's a pain to work in your production environments then the issue is not the technology but the process.
Except one with memory safety and static builds. Like Rust can offer.
The grandparent said "Python is a pain in the ass in production environments".
And the parent said: "any programming language can be a pain in production environments".
My point was how Rust is better for backend services in production, as it wont have memory bugs (which have long plagued backend services and have resulted in many exploits), and it also offers static builds (so deployment can be dead easy, and wont be having the kind of environment issues some languages have).
In the aspect of "design and logic errors", Rust would be like most other languages people actually use for backends, so that's not a factor pro or against.
I think the memory management element is a little dubious, though. Torch’s memory management, for example, is fairly safe, and it internally has its own allocator. A Rust version (I think) that implemented the same would have to rely on unsafe blocks, and would likely have many of the same potential pitfalls as Torch. Time spent satisfying Rust’s borrow checker is likely lost productivity compared to just relying on Python’s reference counting and garbage collection. That productivity could be better spent on making the product better with new features and fixes of algorithmic mistakes.
Could you describe what you mean, as in what that tooling is? Having done some funky deployments in Python before, I've never been very impressed by the ecosystem and tooling.
It implies that all programming languages are equally painful in production, but that obviously isn't the case.
Python is definitely among the worst. Easily the worst among popular languages. It's closely followed by C++. Go and Rust are faaar nicer. Totally different league.
I don't think that is true at all.
Django would frankly be an excellent choice for most web apps and if you're into microservice architecture then frameworks like pantherpy are almost as great as Java's Quarkus. That being said, we do try to limit our Python use to be exclusively for the BI/AI/ML part of our business and that's solely because that's the language those people work with in my area of the world.
This is mainly for pragmatic reasons. We use Typescript for most things because it's nice to have a language that's shared between the front and backend of things when you work in the small teams that are non-tech Enterprise. Then when we need efficiency we use C (and Rust more and more because it protects us from ourselves), when we do embedded it's C and when we need something like Odata or whatever .NET(y) it's C#.
I think Go is nice to write and terrible to work with. Not so much because of the language but because you're going to be writing basically all your libraries from scratch, or alternatively, putting massive maintenance efforts into the ones you consume. Which to be fair, isn't that different from using Typescript, but as long as the frontend of enterprise apps is as it is you're going to be working with Typescript whether you like it or not.
Rust is far from perfect. But in my opinion formed by experience, Rust is the best imperative language available right now.
Note that I have both written software in Python and built a whole devxp stack that includes Python. I wrote the first version of Docker in Python for what is worth, and at my last job; over many years; I built and maintained the polyglot dev experience for 150 devs.
But what about along with the ecosystem/libraries. Would you say in total Java might be better (assuming webdev)? Especially with the Loom project where you don't have to do "async" syntax.
Under this light, Rust has now quite a large ecosystem of pretty good quality I would say.
I have only wrote small thread safe system level libraries in Java. But I sure noticed that javac did not have a borrow checker ;)
For webdev I had no problem with Rust async or not. But my definition of webdev is reduced to a web server that calls out to a database or another service.
This is where having a non-Python PL, such as Rust, can make a huge difference for both ease of production deployment/maintenance as well as having truely efficient production inference.
> Like, really.
Trust me, bro?
> Yeah, but the goal of this isn’t “use this instead of Python to hack up a model”
This was never said.
> You know, those hugging face guys kindaaaaa do know what they’re talking about.
This was never challenged.
> Python is a pain in the ass in production environments.
This is "I don't like python", but phrased adversarially.
Language flamewars suck. Some people don't like certain languages in certain contexts. I see Rust's syntax (having written a small to medium amount of Rust) and feel pretty confident I wouldn't task my data science team with writing or maintaining Rust codebases. We can squabble over "But eventually you'll get it", but ultimately that's more of a priorities comment than a language feature.
But it is an incredible pain in the ass in production environments.
Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e.
One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.
Recently we had mypy stop checking one of our internal packages because of a Dockerfile refactoring. The package could still be found at runtime but the type information just wasn't found by mypy. We didn't notice it until we started getting errors at runtime, which made the fix a lot more difficult to diagnose.
There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started.
For a language which states "There should be one -- and preferably only one -- obvious way to do it" as part of its guiding principles, that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".
Unlikely. If you are using pre-commit to run black then the version being used lives outside your venv and hence not subject to dependencies you are using. Poetry should also be outside your venv.
And, again, if something that was configured wrong this goes back to how managing Python is a pain in the ass. I should be able to dedicate my brain power to the problems I need to solve, not what things need to be in my venv and what things need to be outside of it.
But I'm curious what the nature of a black upgrade being forced on you. Like a manager or stakeholder had an edict stating everyone must upgrade to 23.x?
I feel like your reply is demonstrating this comment perfectly. You could be perfectly correct, but their point still stands that _they_ had a horrible experience that shouldn't have been so difficult. If there's a _right_ way to do things then it should be blindingly obvious and these kinds of experiences would dry up.
> If there's a _right_ way to do things then it should be blindingly obvious and these kinds of experiences would dry up.
This has never happened for any technology ever. People misconfigure things all the time. Even hardened technologies get used incorrectly very frequently. It's just the reality of our industry. Which is why I say, all of this is just a proxy for "I didn't like my experience with Python", which is fine! But also in no way condemning of the technology.
"I find it hard to maintain Python in production environments", but worded adversarially.
> One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.
This is literally just a misconfigured set of tools then.
> There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started.
This is true of all languages and frameworks. There are people _in this comment section_ insisting those that have had negative Rust experiences must simply be doing it wrong. This is more a comment on our industry than any individual technology.
> For a language which states "There should be one -- and preferably only one -- obvious way to do it"
Yes, this has always been a bold faced lie from the Python community. Marketing speak is marketing speak.
> that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".
It's as much a pain in the ass as literally every piece of technology, as the above statement is true for every technology and framework that exists.
I acknowledge that you don't like it. That's great! But again, language flamewars suck, and suggesting that something is bad because you've individually had a bad experience is pretty silly. Lots of people successfully deploy and maintain python applications of various sizes without referring to the process as "a pain in the ass".
>> This was never said.
Quotes don't have to used solely to indicate somebody explicitly said something.
Personally it's my most productive language, and this is after years in Python, NodeJS, and ~5 years Go.
Which isn't to attempt to invalidate your experience. But i do think:
> writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.
Is quite a leap. A language you describe not speaking natively is, unsurprisingly, difficult for you. Which is a far cry to say it's always that way.
You of course shouldn't be in a rush to rewrite anything unless you see the value in it. I make no claims of that value, either. Rewrite It In Rust (RIIR) is a meme, imo a destructive meme. But i do take issue with labeling any language broadly.
Most languages are pretty good when you're very fluent in them, and while they do have tradeoffs, i don't think Rust's tradeoffs are as you put them.
No killer app here. We just prefer Rust itself. Go is very capable and a great choice.
For me personally i found Go to spread logic out far too much. Some features of Rust, Iterators for example, massively simplified how i think about data flow. That was my #1 issue with Go. This has side root-causes in Go of course, like no generics to allow for Iterators, etc - but these days Go has generics so it's likely improving. Features around Enums are also huge for me.
I left Go ages ago and it's improved a lot sense then. I still prefer Rust, though.
As i said no judgement on any lang, including Go. This is merely my experience and preferences.
In general, I would probably reach for Rust if I needed to do anything wither Web-Assembly (wasm) as the path to working is much nicer than other languages imo. Along those lines, Yew is pretty nice as a rust-like framework.
I've also played with Rocket, and found it very nice and effective for api driven web-apps. Haven't done any templating engines in/with Rocket, however.
Arguing "Rust has no dev speed penalty" in that context is just... no. You are damaging your own cause there, because all advocacy becomes laughable.
While yes, that's really bad, it's just temporary build files. The actual executable should be fairly tiny.
From another response, their Windows installer is under 7mb, and much of that is MSI boilerplate.
Rust embed is also great, I cross compile everything into a single dynamically linked binary and scp to my vps, deployment is a breeze.
https://github.com/tokio-rs/axum https://tokio.rs/tokio/tutorial
That is incredible. It's like being able to carve- and install a door from a slab of wood faster than installing a prefabricated door.
It's one of the reasons why Python developers don't like working in Ruby, for example.
So the claim doesn’t sound so crazy to me. For me at least, types remove mental overhead, they don’t add to it. I can’t count the times I’ve been reading through untyped (or inaccurately typed) python code and have had to jump a dozen levels deep just trying to figure out if a function can or cannot return null.
In my personal experience, once you're proficient in Rust, the initial cost is not somewhat high because those improvements in your ability to reason about your code base start snowballing quickly. And the long-term effects are enormous. I have literally picked up a project I hadn't worked on in years, made sweeping changes to the internals, and had every test pass in a pretty comprehensive suite the very first time it compiled.
After each try with Rust, I ended up going back to c++ myself, it's just more productive in practice.
As to falling back to any given language, that's not uncommon. Learning a new language is difficult, especially if the paradigms don't align well to what we already know, or have experience in. This may come down to experience, or it may come down to how you now think through problems. That said, I don't think that C++ is inherently more productive than Rust, I think saying your experience makes you more productive in C++ is a far more likely truth.
That has been my experience as well. Rust and the borrow checker get too much in my way. If I want a high level language I will use Python but if I want control all the way down to the hardware C++ is where I go. Plus, it /feels/ that ISO C++ has significantly picked up on momentum and is seeing more investments than Rust.
Have you tried "sticking" with it for a while? Try forcing yoursel to use it[1]. I find that's the best way to learn it (and try reading more code from open source project in Rust you like).
I have an opposite experience writing Rust: it allows me to focus on the problem more easily, especially when the problem domain is complex and the codebase grows.
[1] a trick often used while learning human languages
I even like Rust, a lot. I really love things like how you actively need to make things mutable. I wish that was a standard feature in any language I work with. But it's just so verbose.
Not the person you responded to but yes. I have written a lot of production software in Python. I remember going developing APIs with Python and the day we switched to using Typescript all these bugs we used to have just disappeared. Going from Typescript to Rust was not as big a change but well designed statically typed languages are a productivity booster not killer.
We can agree to disagree but I've personally experienced a large Python codebase at a Unicorn tech startup and I'd like to never experience it again.
I've been exposed to Python through work, but I was never a fan. Backwhen I was faced with learning Ruby or Python, I picked Ruby; just felt more ergonomic. I even prefer more recent versions of TypeScript to Python too. I never really felt productive in Python, since the language is very finnicky (I would say more so than JS).
Rust might be verbose in parts of the language (but I don't find it as verbose as C or C++), but you get a lot of goodies the the price of a bit of verbosity. So, I don't mind it
I think it's worse than C. That being said, I'm probably also biased in a way where I'm more forgiving to C++ than to Rust because C++ is old and Rust is new. What I found when I was working with Rust was that I wished you'd write it like you write Go. As for Go I quite quickly wished it worked like Rust. So I guess you can't win with me?
These days we mainly use Typescript because it's useful to mainly use a single language in small teams since it allows the one frontend guy to go on vacations without a laptop. We still do some C#, some C and some Rust + the Python which runs our ML/AI/BI parts since that's what those people work with in my area of the world.
It's likely that they just haven't got to the point in their programming where they understand the need for the things Rust solves. Like if you only ever write 100 line Python scripts you probably haven't run into the performance issues or difficulties that come with working on large dynamically typed codebases.
Also, I'd like a pony, a jetpack, and VIP passes to the premiere of Dune: Part 2.
Haskell is a bit of a tricky language organizationally for mostly social/political reasons, but if I end up starting my own company in the future, Haskell + Rust is definitely the tech stack I'd lean towards.
That said, we were one of the largest Rust shops outside of the FAANGs, but Rust was straightforward to teach to C/C++ folks.
1. A lot more people want to do Haskell than there are Haskell jobs
2. Haskell is an easy way to signal (show not tell) that our team is willing to do things differently—especially important because our team was at Target, which did not have much of a reputation as a tech employer!
We didn't try hiring Rust programmers because the existing Haskell folks were more than happy to pick it up.
So making hiring easier is one of the reasons I'd want to use Haskell, counterintuitively!
I'm glad it worked out for both of us.
I do a lot of AI Engineering in Python, but production Data Engineering, and Feature Engineering workloads are done in parallel with Rust. Simple and Fast.
You can even make custom-made python modules with a Rust core when Numpy matrices are not enough to crack your graph-traversing problem efficiently ! (Thanks PyO3 and Maturin)
Not even. Some people start building big systems with Java or Go. It's entirely appropriate to start in Rust too once you're more proficient in it.
For anything that is not related to DS/ML/Opt, yes, start with Rust. I had tremendous success with distributed Rust microservices using ZeroMQ for backbone communication.
You get operator overloading, so you can have ergonomic matrix operations that are typed also. Processing data on the CPU is fast, and crates like https://github.com/EmbarkStudios/rust-gpu make it very ergonomic to leverage the GPU.
I like this library for creating typed coordinate spaces for graphics programming (https://github.com/servo/euclid), I imagine something similar could be done to create refined types for matrices so you don't do matrix multiplication matrices of invalid sizes
It's like this until it isn't. Eventually this falls completely away - I promise.
It's a lot like being a 100% backend engineer trying to develop in React for the first time. You just have to get used to it.
Rust is not difficult, it just takes some soaking in.
I like Rust, but Mojo seems like it ought to be a much easier sell for projects that are very invested on the Python side.
Like if your build server is backed up maybe the rust version of your language's compiler and interpreter will make your build server not be backed up because it does everything faster
But for everyone else its not that useful to be using the rust version of everything
Its just another one of those things where you see lucrative job offers requiring rust, so you make up reasons to have impactful-ish experiences in rust
- Copy structs- ownership issues vanish when things can be bitwise-copied
- .clone()- don't be afraid to clone outside of hot loops! If allocations are a problem, look at using Rc instead of Box where applicable
- Pure functions/methods- you don't need mutable references if you aren't mutating things
- Macros- reduce boilerplate
- Purpose-built traits- you can hang methods off of different structs that can make certain flows really ergonomic
In general: Rust asks you to care about all the little details at a language level, but it also empowers you in different ways to build high-level abstractions on top of those details so that you often don't have to care about them. I'm finding that I have the smoothest time with it when I use traits and macros to make my own little DSL for the problem I'm trying to solve, and then write my business logic in that.
And then- many libraries offer you their own abstractions that give you the same sort of benefit. Concurrency normally involves a bunch of ownership and locking headaches, but something like rayon makes it breezy by giving you an abstraction that handles those details for you. Async web route handlers with shared state from scratch would normally be hard, but axum and Rocket give you abstractions that handle the gross stuff for you. One of my favorite crates I recently discovered is just called "memoize", and it just gives you a macro you can slap on any old function to memoize its results! (global mutable state!) https://crates.io/crates/memoize. This would be a mess to implement yourself, but the abstraction makes it breezy
And then personally: the main reason I'd use a Rust library like the OP over a Python library is the tooling/dependency management. Running a Rust project with dependencies Just Works and that's extremely valuable to me. But, it'll always be weighed against the costs of using a lower-level language.
In other words I am more willing to trust abstractions over tricky problems in Rust than I would in other languages.
Just for a second, try to imagine a generic memoize in C++.
But I hope to get a suggestion for switching to the ext language in the real life. Like I’m quite proficient with swift language, I’m making money with it for years, so I consider my proficient comes not just from general computer science knowledge, but it comes from the getting o know the environment as well. Like I known the sdt lib, I know common caveats with the project, I have a pretty much experience it’s it after all.
And so I’ve tried rust. I mean it’s fantastic, it have awesome tooling, concise syntax and well crafted design, but I barely could write script with it. And I’m seeing clearly that this will lasts for a year or something, and even then I’ll suck at it towards an average senior dev focused on it a way beefier me.
So I really can’t crack it myself, like how do you folks switching to a new lang to get paid for it?
Personally I don't think it's worth trying to predict which technologies will be the most valuable and learning them based on that, unless you're struggling to make money right now. It sounds like Swift is working great as a money-maker for you right now, so I recommend just following your interests outside of that! Maybe they'll lead to another money-maker, maybe they won't and that's okay
I don't see Rust eating any domains outside of low-level systems development anytime soon, so I wouldn't worry that you're going to get left behind or something. It's not going to make Swift, or JavaScript, or Python, or Java, or Go, or many other things obsolete
I also really like that when I use libraries, I know that all the library authors also subject to these same restrictions and I can trust their code much more than I could in other languages.
Looking back, it seems to me that Python's ecosystem fractured with the 3.0 release, creating a dynamic where other languages and their ecosystems had better vitality and benefitted from more improvement over the last decade. The ML development and other research that has happened in Python are clear exceptions -- these have been world-changing -- but it's disappointing for me to swallow Python to use ML libraries. I have to say, I'm more than enthusiastic to see alternatives emerging.
I think it's becoming clear that Python is to ML what JS is to the Web. We got stuck with it through a series of historical coincidences, and no one is super happy about it. But it mostly does the job and has evolved to fit the bill.
I'm not a huge fan but it's definitely here to stay. Hopefully Rust can take off but I don't see any other viable replacement.
I am hopeful that languages like Mojo and Val are going to find cleaner ways of doing what Rust does best, and that Rust can learn from them.
I suspect they'll be great for a subset of the things that makes Rust great. There's probably a bunch of essential complexity that can't be hand waved around via a "smarter compiler" at least at this point in time (I'm not saying Rust is as simple as it could be, but I suspect there's no order of magnitude improvement while fulfilling the niche that it occupies).
Typically the people I hear saying that are super Spring indoctrinated.
I am not an expert in rust, but looking at it as potential choice for my next project.
My problem with Rust, now on month 3 of trying to love it, is exactly what you mentioned. It's a slog. Comparing it to other languages you have to know a lot more about systems. Initially, this isn't a problem, but as it turns out it becomes a problem with Rc/Arc/Box/etc. These are really just leaky abstractions in practice with strange behavior in many cases. Sure, you can write Rust without ever needing that. But if you ever need something more specialized than BTreeMap you're going to be chest deep in a kafkaesque nightmare of lifetimes, weird garbage collection, crazy types (Option<Box<Rc<Cell...) etc. It's at this point the helpful compiler becomes a mortal enemy. A sisyphean task made worse by the compiler suggesting the wrong things at the wrong time because you're so deep in lifetime/box/[A]Rc hell that not even God himself can save you.
As long as you can mostly use the standard library and mostly avoid ever needing to do something where resources are shared Rust isn't that bad. Once you start sharing resources your life becomes extremely painful extremely fast. In some cases this is good, requiring you to think, but in many cases it's bad due to the leaky abstractions over memory when you need to do something more. In this case, despite it's major flaws, C++ wins in my opinion. I have a problem with the abstractions over pointers in Rust because they require me to STILL know how things underlying those abstractions works.
Rust is going through its 15 minutes of fame. Much like Ruby, Elixir, etc you will have fans trying to shoehorn every possible case into it. It's a systems language. It is very good at what it is. It definitely does not belong in Data Science though it could be used to replace the underlying systems-level code found in Tensorflow/Torch/Pandas/etc a little safer. But at what cost?
I'm being hard on Rust because I think in 5 years time it'll begin to replace C++ for most new systems level code. I hope they continue to revise the API.
https://www.infoworld.com/article/3267624/whats-new-in-the-r...
Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case...
I really want to like Rust - I've read the Rust book, done the Rustling exercises, read Zero to Production in Rust and did the project there, and still it's not clicking - writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.
So I'm not in a rush to rewrite anything written in Python just yet.