Re:Mypyc: Compile type-annotated Python to C
by throwaway894345 on Tuesday February 23, 2021 @04:46PM
(#26243322)
Scientists aren't the only players at the scientific computing table these days. There's increasing demand to bring data science applications to market, which implies engineering requirements in addition to the science requirements.
> In my view, Python needs to implement a JIT to alleviate 95% of the need for Numpy.
Numpy is just statically typed arrays. This seems like best case for AOT compilers, no? I'm all for JIT as well, but I don't have faith in the Python community to get us there.
Re:Mypyc: Compile type-annotated Python to C
by jessermeyer on Tuesday February 23, 2021 @05:45PM
(#26244013)
JIT works great here too. It would see iteration and the associated mathematical calculations as a hotspot, and optimize only those parts, which is easy since the arrays are statically typed and sized.
I say this as a Computer Scientist at NASA that tends to re-write the scientific code in straight C. But for many workloads, a JIT would make my team more productive, basically for free as a user.
Re:Mypyc: Compile type-annotated Python to C
by throwaway894345 on Tuesday February 23, 2021 @06:51PM
(#26244677)
Yes, JIT would work well also, and I would strictly prefer a JIT, but I don’t think we’re likely to see a JIT Python with good ecosystem compatibility in the next decade. Good luck to the people who are using Python these days, but I’m tired of fighting the same major problems we had 15 years ago. Other ecosystems solved those problems and they actually improve materially.
Re:Mypyc: Compile type-annotated Python to C
by pjmlp on Wednesday February 24, 2021 @02:00AM
(#26247378)
That is why I am so much into Julia, even with its adoption bumps.
The problem is not that Python lacks JITs, rather the community culture of rewriting code in C instead of contributing to JIT efforts.
Personally I just use a JVM/.NET based language, and if I need I can use the same C, C++ and Fortran libraries that Python uses anyway.
Re:Mypyc: Compile type-annotated Python to C
by johnnycerberus on Wednesday February 24, 2021 @04:32AM
(#26248220)
Julia was created to tackle problems in applied disciplines (physics, neuroscience, genetics, material engineering, etc.). I was expecting it not to be picked up by your everyday app developer or by the overly abstract functional programmer. As an afterthought, personally I think Julia can do much more than that, I would say it can do at least as much as Python is capable today, but better.
Re:Mypyc: Compile type-annotated Python to C
by pjmlp on Wednesday February 24, 2021 @04:48AM
(#26248303)
The ecosystem is slowly expanding beyond applied disciplines, because when those people need to code something else, e.g. a Web site for their research data, then as usual they try to use the hammer they already know.
Re:Mypyc: Compile type-annotated Python to C
by throwaway894345 on Wednesday February 24, 2021 @02:23PM
(#26254363)
I'm really interested in Julia's performance for general purpose application development. It's great that it can work with large numerical arrays very efficiently, but what about large, diverse graphs of small objects like you commonly find in general purpose application development? I think I want a hybrid between Julia and JVM or something.
Re:Mypyc: Compile type-annotated Python to C
by nerdponx on Wednesday February 24, 2021 @10:03AM
(#26250589)
Does your team use Numba?
Re:Mypyc: Compile type-annotated Python to C
by JesseMeyer on Thursday February 25, 2021 @08:29AM
(#26262494)
Where able, but its poor treatment of SWIG makes interfacing with standard tooling a royal pain. In many cases, I've rewritten Numba code in Cython or C for this very reason.
C = A + B
Python with Numpy perfectly service just that need. We all have our grief with the status quo, but Python needs data processing acceleration from somewhere. In my view, Python needs to implement a JIT to alleviate 95% of the need for Numpy.