You mean Dyalog APL? It already regularly outperforms normal, handwritten C code. It also has support for distributed computing, multi-threading, and the Co-dfns compiler can be used to compile your APL code to the GPU. For example, consider the following talk, which discusses sub-nanosecond lookups/search using the Dyalog APL interpreter.

https://dyalog.tv/Dyalog18/?v=paxIkKBzqBU

Supposedly numpy was inspired in part by J.

I don't think it will ever dominate the market - some things such as FFI remain weak points compared to C.

Plus a lot of what makes Mathematica good is that they have specific algorithms that they've optimized quite a bit, for things like computer algebra &c.

As mruts pointed out above, kdb+/q provides what you're imagining - it provides fantastic distributed computing support via its IPC protocol and also has multi-core support.

Multi-core is priced for big financial institutions, though. Only thing you or I could afford is the 32 bit single-core.

There is a free 64-bit version available with limits that don't seem harsh at all:

The 64-bit kdb+ On-Demand Personal Edition is free for personal, non-commercial use. Currently it may be used on up to 2 computers, and up to a maximum of 16 cores per computer, but is not licensed for use on any cloud – only personal computers. It requires an always-on internet connection to operate, and a license key file, obtainable from ondemand.kx.com

You can start the 32 bit instance with the -s switch (for q to be started with multiple slaves) for parallel execution of a function over data using the "peach" command [0].

But yes, I do agree that the 32 bit instances have limited use, as they can't be used in for-profit projects.

[0] https://code.kx.com/wiki/Reference/peach

Is the pricing published? I haven't been able to find it on the website.

It’s definitely one of those “if you have to ask, you don’t need it” kind of things