News for hackers. Stuff that matters.
Re:The GIL and its effects on Python multithreading by fastasucan on Wednesday September 29, 2021 @05:00AM (#28691631)
You cant both claim that it is major language for complex data analysis and that it is "unsuitable for large parts of the job". If it is a major language for complex data analysis it is apparently not unsuitable. It can be not optimal/not the best/bad but it cannot be unsuitable, because then your first statement isn't true. Which one is it?
Re:The GIL and its effects on Python multithreading by lmm on Wednesday September 29, 2021 @05:04AM (#28691659)
It's a great language for exploratory prototyping of complex data analysis and a terrible language for robust implementation of complex data analysis. Well-managed organisations prototype in Python but take the time to rewrite the prototype for production use; poorly-managed ones skip step 2.
Re:The GIL and its effects on Python multithreading by pharmakom on Wednesday September 29, 2021 @07:13AM (#28692261)
Does this seem really inefficient to anyone else? Why is there not a language that can do the prototype and offer a high performance ceiling? Julia seems like a candidate.
Re:The GIL and its effects on Python multithreading by penteract on Wednesday September 29, 2021 @11:55AM (#28695439)
There's a language design tradeoff - Python is is the result of the designers almost never saying "We should do this to make Python faster, although it would make the language slightly harder to program in."

This even goes as far as choosing not to make improvements that improve performance but make the implementation more complex (for example adding a JIT), which allows Python to continue to add language features.

Re:The GIL and its effects on Python multithreading by lmm on Wednesday September 29, 2021 @09:01PM (#28702033)
The language isn't the hard part. One of the more programming-oriented data scientists was actually happy to write Scala to start with, but honestly getting from a prototype to a production-quality implementation is pretty much a rewrite anyway (e.g. you have to consider all the error cases).
Re:The GIL and its effects on Python multithreading by FuckButtons on Wednesday September 29, 2021 @03:25PM (#28698610)
It doesn’t work like that in any other industry, why should it work for software? Mechanical engineers don’t put prototypes into production, why should software engineers?
Re:The GIL and its effects on Python multithreading by pharmakom on Wednesday September 29, 2021 @06:26PM (#28700788)
we are writing software, which is a totally different category to mechanical engineering.
Re:The GIL and its effects on Python multithreading by maccard on Wednesday September 29, 2021 @05:11AM (#28691697)
What language do you rewrite it in so?
Re:The GIL and its effects on Python multithreading by chalst on Wednesday September 29, 2021 @06:01AM (#28691906)
It's typically pretty easy to turn Python+numpy into performant Julia.
Re:The GIL and its effects on Python multithreading by rofrol on Wednesday September 29, 2021 @10:47AM (#28694367)
I remember reading this comment:

>I recently tried to migrate my R code to Julia. Even though I already knew R data.table is faster than DataFrames.jl, I was totally blown away by how slow Julia is. So I quickly gave up. I think I will have to write unavoidable hard loop in cpp, which I really don't want to do...

https://news.ycombinator.com/item?id=27404910

Re:The GIL and its effects on Python multithreading by pjmlp on Wednesday September 29, 2021 @12:33PM (#28696067)
Apparently it is working for some, https://juliacomputing.com/case-studies/
Re:The GIL and its effects on Python multithreading by lmm on Wednesday September 29, 2021 @09:02PM (#28702037)
In the case I was thinking of, mainly Scala (we were a very JVM-oriented place and used Spark clusters for our big data stuff).
Re:The GIL and its effects on Python multithreading by nick_ on Wednesday September 29, 2021 @09:54AM (#28693700)
They're saying that Python has gained a reputation for being the language for complex data analysis, but in fact it is objectively "unsuitable to large parts of the job".