I'm a data engineer at FAANG. I love the data scientists I work with. They are, generally speaking, crazy smart and simultaneously humble about their (in)ability to write code - they're highly specialized in ML, not so much SWE. I therefore have generally good job security working in operationalizing and optimizing their code. (Recently tweaked a script a scientist wrote and dropped runtime from 5hrs to 5min.)
If I never saw another line of Python again, I'd probably be quite happy. But I think they - and even some ML engineers I work with - love them some Python precisely because they can mash the keyboard a bit and take the shortcut to the finish line. And I don't begrudge them this at all - good on them that they can get their job done quickly! But it's a pain to make things stable and efficient.
Some of the biggest distributed systems built today are used for statistical inference or scientific computation
Most "ML people" I know are highly versatile in software, networks and deep hardware knowledge, i.e., essentially they have a very good understanding of what a computer is and what is capable from
Its very naive to think that you can assemble machine learning systems without having a solid understanding of computers and statistics
You know who also likes python a lot? Hackers. I wonder why
https://github.com/python/cpython/blob/main/Lib/multiprocess...
In the begging is very simple to churn out code and do wtv you want, but the more you are into it, you start realizing that there are endless possibilities
It's a great language for beginners and even better for experts that just want to solve problems with code without thinking to much about if coding is beautiful or not, or feeling cool, or arrogant about it
It just works
Also monkey patching isn't unique to python.
JavaScript it didn't work:
class testClass {
constructor() { }
callHoHe() {
console.log('ho', 'he');
}
}
let hi = new testClass();
hi.callHoHe();
testClass.callHoHe = () => {
console.log('haha');
}
let heh = new testClass();
heh.callHoHe();
For Java people didn't think it was possible:
https://stackoverflow.com/questions/47006118/is-there-any-wa...
For Java they said here that you just have to use your own similar implementation.
And for C# they have some pretty intense restrictions on overriding standard library stuff:
https://stackoverflow.com/questions/21302768/where-we-can-ov...
Golang doesn't seem to have this functionality as well:
https://stackoverflow.com/questions/37079225/golang-monkey-p...
Ps. it would have been nice to have monkey patching when dealing with btoa and atob in JavaScript, since they have different function on NodeJS vs the browser.
The better solution is to encapsulate the class and override the methods. Monkey patching is terrible because the behavior of the function is changing at run time. If someone is not aware that you are monkey patching a function the only way for them to determine what is going on is to step through the code with a debugger.
Isn't that true of basically every language supporting class-based OOP and inheritance?
class.methodName = newMethod
.. kinda thing, future callers now get your method instead of the original.
This does seem a fair bit easier than other languages make it to do?
class.methodName = newMethod
Which is as powerful as it is a problem, since doing such kind of monkeypatching will change the behaviour all other instances, including already-created ones, that know nothing about your trick.
Any part of the program can modify any other part of the program in a significant way, making local reasoning and debugging very hard.
So, great for quick-and-dirty single-file scripts/ipython notebooks. Terrible for large systems.
That's the very issue with Python. The way it doesn't enforce sane, clean programming behaviour makes it easy for a beginner/non-programmer to work with it. But a large system with a lot of external libraries is very hard to maintain.
Source: Python user since ~2004
Inheritance and overriding method in the descendant class is cleaner, and more broadly supported. When you need monkey patching, sure, its nice that most modern dynamic OO languages support it quite naturally. (Ruby even supports scoped monkey patching via refinements, as well as classic monkey patching and per-object overrides.) But this is not at all unique to Python.
It's sexy and most of the times ...
I'm sure every single one of them is capable of writing world class code if they feel like it
Their exploits are world class and their focus is to exploit
I have seen stuff in JavaScript exploitation that I can't even scratch the surface. I feel like I have been playing piano for 15 years and I can't even understand if that a music that is playing
Almost always, ML production models end up being a binary files of matricial weights. This file can be loaded in wtv language or device you decide to use
The tiny subset of people who build ml systems (say tensorflow core devs, write actual distributed systems, etc) are actually hpc specialists, and have all the qualities you describe.
Of course, you may work somewhere where you're lucky enough to have everyone be good at everything!
I have experience working alongside ML people and interviewing them, and have to agree (anecdotally) that this is often the case. Not only that, but they often do not make it a priority to get good at these things.
Most ML people there cannot build large robust systems and some struggled with the non-algorithmic bits of software. I am sure that some can out there in the world, but for the most part our ML people were very good at creating models and not very good at the development part, especially as the program grew (part of the motivation to hire devs like me in the first place).
Python gets rid of as much of the developmental complexity as possible. No types, no memory management, libraries for everything, No need to create a class to run "hello world." Pip makes it trivial to import things. Use PyCharm and you just need to click the run button, with no complicated JRE and JDK setup.
It is the fastest way to start writing models.