News for hackers. Stuff that matters.
Re:R adds native pipe and lambda syntax by smabie on Saturday December 05, 2020 @07:19PM (#25319738)
Pandas/python is amazingly prevalent at trading firms. And everyday, we bitch about the performance, we bitch about the stupid API, we bitch about the GIL, the lack of expressiveness. The list goes on and on. But for some braindead reason, we never switch to Julia. It's masochistic.
Re:R adds native pipe and lambda syntax by ivirshup on Saturday December 05, 2020 @10:45PM (#25320834)
I do think Julia is a far better language for numerics than python, but compared to DataFrames.jl, pandas can be quite fast. I know, "but it's easier to make it faster in Julia". Last I checked `sort(df, :col)` was significantly slower than `df[sortperm(df[:col])]`. Someone actually has to go through and make these libraries fast.

Second issue, in my field (bioinformatics) the script is still a pretty common unit of code. Without cached compilation being a simple flag, Julia often is slower.

Re:R adds native pipe and lambda syntax by smabie on Saturday December 05, 2020 @10:51PM (#25320861)
Yeah, that's a good point. DataFrames.jl starts to really shine what the cookie cutter pandas functions arent adequate for what you need to do. DataFrames.jl can certainly be slower in some cases, but you should expect a consistent level of performance no matter what you do. This is a farcry from Pandas, which tanks by large factors when you start calling Python code vs C code.

In regards to Julia's compilation problem, you can use https://github.com/JuliaLang/PackageCompiler.jl to precompile an image, allowing you to avoid paying the JIT performance penalty over and over again.