Python addresses a good deal of the problem with Cython, which lets you declare variables and compile to get ~90% of the speed gain you would get from a pure C implementation. And then you can just link to a C implementation if you still find you need to build one...
I think what's far more common is prototyping something in Python or R, and then realizing that 99% of its CPU time is already spent in wrapped C code. And then you call it a day and move on to other things :)
This is why Julia hasn't taken off nearly as quickly as I initially expected - Python/R/etc are highly optimized for their common use-cases.
Someone has to write those libraries.
> This is why Julia hasn't taken off nearly as quickly as I initially expected
Not sure what you were expecting, but consider:
- the most conservative userbase estimate I would believe is 20k users (based on mailing list subscriptions, website stats, and download numbers).
- sustained package ecosystem growth: http://pkg.julialang.org/pulse.html
Yes, but the discussion is about the "extremely common" case of prototyping, which definitely should not require library building, and usually should not require veering much from established libraries.
>Not sure what you were expecting
The data community can coalesce around a tool extremely quickly, in the matter of a year or two. Spark is about the same age as Julia, and has a thriving ecosystem around it. In 2004 R was a fairly esoteric analysis tool, but 2008-2010 it was the de facto data science language. Python made similar advancements in just a few years in the data science community.
Julia? I don't know a single person who uses it day-to-day, but I know a lot of people who tried very hard (myself included). The critical mass simply is not there: people aren't building packages because the users don't exist, and they don't exist because the packages don't exist. Your chart shows a linear growth in packages, which implies a constant amount of development work. This means it's not a growing language. This is what a growing language looks like: http://blog.revolutionanalytics.com/2010/01/r-package-growth...
This could be early exponential growth.
Or the constant growth may hit a tipping point as a critical mass of tools and infrastructure is developed.
Python is old and stuck with some serious design flaws which can not easily be fixed. The question is, can Python get fixed in shorter time than it takes to give Julia a competitive set of libraries.
For me, there is not something I can think of being such a deal breaker for me to switch to a another language. More likely it might not be ideal but could addressed by the ecosystem of Python itself.
Because Python has so many C/Fortran external dependencies.
Is Julia going to change it? By itself maybe, but currently it still borrows from Python in order to enrich its ecosystem, so basically adding even more layers. That for me, speaks to the volume that language is not really the pain here, library is.
Real-world problems are ugly deep down to its core and the requirements and constraints are every changing. Who thought GPU programming will be such a big deal just 5 years ago, if not DNN revolution completely turning thing around? And I don't think Julia's improvement is really prepared for this. Even more, if Tensorflow style data flow programming is better accepted, then probably any language could be used to just draw some the computation graph and the heavy lifting will be done by the framework anyway. It is going to be harder to persuade programmers to adapt a language just for the perks that is only peripheral.
This is a consequence of how easy it is to interface with languages like C in Julia. I.e. whenever a new CUDA library or change happens, it will take the Julia devs less effort to update their wrappers.
Add to this Julia's HPC support and it is by far the best offering if you want to do REPL style programming on a GPU cluster.
I must take issue with "near complete" - CUSOLVER.jl, my other CUDA wrapper, is missing a lot of the RF (refactorization) functionality and doesn't really have a pretty high level API yet. The doc and testing situation is also pretty bad. Everyone else's packages in JuliaGPU (the overarching GPU org on GitHub) are in a much better state.
You are right, though, that writing CUDA bindings in Julia is very easy - so easy that I can do it! It's also thanks to packages like Clang.jl, which make it easy for us to automate the procedure of wrapping the low-level interfaces.
Finally I must add the caveat that although I should test CUSPARSE.jl and CUSOLVER.jl with MPI (and GPUDirect) I've been extremely busy recently and not able to do so. If anyone wants to help out in this regard I would be very appreciative!
I've been using the remotecall functions rather than MPI (bunged my code into ClusterUtils.jl). Don't know how I'd use GPUDirect with this strategy.
The packages based on CUDArt.jl all seem to work with remotes. So long as you don't fetch a pointer to memory on a remote device.
Once the wrappers are done, for most developers, it becomes the matter to layering those build blocks building their own functionality. It is not a strong enough argument that people should use Julia because it is a better language to write wrapper. Let alone that Python wrappers for data science stuff is already like a de-facto builtin.
Not saying Python is perfect, but IMHO, Julia is not hitting the right spot in order to really stand up against Python.
Everything described here, for starters: https://news.ycombinator.com/item?id=11522767
(also: http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to...)
I love Python (despite the community). Sorry to be that guy but Python is always the second best and for me that usually means I end up using Python less and less. There isn't one area that Python is the best tool and sadly that is why it just isn't taking over the programming world.
As the author says, the biggest drawback is libraries, and that is (1 of) the big attraction(s) towards languages.
Nobody wants to have to switch between C, Python, Assembly, etc. to make their code faster, but even fewer people want to write bindings to underlying C code. This also goes exactly against what the author doesn't want to do, which is to yak-shave around the language to get things working.