For the past few years I've been telling people about the APL family of languages. The clincher for me was seeing the video of an APL version of Conway's Life mentioned in the submission. The impressive part wasn't its brevity, it was how they approached the problem.
In APL you have a 'rotate' operator that shifts data in a particular direction. If you have a vector and you rotate it once, every element moves to the next position and the last element then becomes the first.
The nice thing about APL is that most operations work on data regardless of its dimensionality. So, to do Conway's Life, you take your 2D matrix of cells and produce rotations of it in eight directions (N,NE,E,SE,S,SW,W,NW). You then take those rotated versions of the matrix along with the original and conceptually stack them. Then, for each grid point, you sum downward, producing a new matrix that contains the neighborhood count of the original matrix. From that you can create the next Life generation.
This sort of problem doesn't come up everyday, but the thing that I think is profound is that the existence of these operations allows us to think about problems in different, possibly simpler ways. They are untapped potential and they could be as well known as map and fold.
APL and its derived languages are hard to approach but there isn't much that keeps us from importing the data structures and operations in more approachable languages.
Do you believe that refactoring a large APL codebase would be inherently easier than something like, a large legacy C++ codebase?
What is the relative complexity of comparing, say 1000 lines of APL to 10K lines of C++?
I remember reading an interview between Arthur Whitney and Bryan Cantrill where they mentioned something about recognizing idioms in the dense code much more easily in K than in something like C because it took so many less characters. Do you see any sort of refactoring advantage to that?
I've also recently seen work on the Co-dfns compiler in APL where the author mentioned that APL allowed him to easily refactor compared to traditional codebase, and looking at the github library, there is something like 3 million edited lines over the span of several years in a codebase this is a few thousand lines of code. I find this fascinating because my hope is to eventually start a small software business and I think reducing complexity in the number one priority in order to make it sustainable in the small (the goal isn't to have a team of 50 developers).
Thoughts?
I think array languages, or at least their operation set and idioms, will move into the mainstream in the same way that functional is, but it will take time. Sustainability for your business would be more an issue of hiring and retaining talent.
Reach out if you want to talk about this more: @mfeathers
APL makes you see and envision computational problem solving differently than, say C. Use it enough and your brain starts thinking in patterns, shapes, vectors and matrices being "zippered" together or apart, contorted and distorted.
Take a matrix of values and hit it with another matrix of 1's and 0's to select elements and then flip it, slice it, convert it to a vector and combine it with another vector to form a new matrix, then scale every element and apply a polyphase FIR filter across the rows using another matrix for the coefficients to then arrive at a vector containing your answer. This is made-up, of course, but that's how you start thinking, you start seeing parallel computing in your head and you can reach for it instantly with a notation that allows your hands to describe what you just imagined with a few well-selected symbols. No other language sends you on these flights.
It is truly seeing and approaching computational problems from a very different perspective. And the notation makes it comparable to composing music rather than going through the ugly mechanics of text-based programming languages.
I find it hard to explain to those who have never experienced this revelation of sorts. Watching a few videos and playing around with APL isn't enough. If I had to guess I'd say someone would have to work with it on a daily basis for about a year to achieve that mental shift.
Again, no different from musical notation. It is impossible to explain or experience how you see music once your brain can look at a page full of funny symbols and your mind starts to hear the music. The relationship with the piano, guitar or other instrument changes once that mental connection requires no thought at all.
Getting there with APL takes time. For example, you need to be able to sight read idioms like thee following much as a musician learns to read a two hand chord and instantly place their fingers exactly as required on the piano, without though.
http://docs.dyalog.com/14.0/Dyalog%20APL%20Idioms.pdf
or a large enough subset of these:
http://aplwiki.com/FinnAplIdiomLibrary
Back then, a lot of us used to walk around with a small pocket-sized FinnAPL idiom book in our pocket and had the FinnAPL idiom library book on our desk.
For people interested in Byzantine music, the notation can seem imposing at first -- especially to people who already know western notation. Much effort has gone into translating Byzantine music into western notation, but some problems arise. First, some Byzantine scales use very different pitches compared to a western equal tempered scale. Second, the rhythms in Byzantine music can involve syncopation or other rhythms that can be difficult to notate using western rhythmic symbols. Finally, Byzantine notation is optimized for depicting melody, while western notation is focused on harmony. There is simply more unnecessary visual noise in writing Byzantine music in western notation.
That last point resonates with your talk of idioms. While Byzantine music may seem to have many neumes for a given musical phrase, there are many idiomatic combinations of pitch-changing and rhythmic neumes together. As such, what would be read in Western notation with many individual notes and rhythmic markings, becomes in Byzantine notation a single "word".
If you're interested in seeing how these systems of notation compare, look at the following hymn: http://www.cappellaromana.org/wp-content/uploads/2014/04/Che... The neumes are in red at the top, with a simple western interpretation on the first western music staff and an ornamented interpretation on the bottom staff. Comparing the neumes to ornamented western interpretation (which represents the full Byzantine ethos, or style), we can see that significantly more information is contained in the sparser Byzantine neumes.
Anyway, you've convinced me to learn APL because I now see how similar the arguments for this notation are to my own arguments for using Byzantine notation. Thank you for your insight.
Here's a paper by Ken Iverson about the power of APL notation:
http://www.eecg.toronto.edu/~jzhu/csc326/readings/iverson.pd...
Why does J exist?
Back in the mid 80's (I don't remember exactly) dealing with APL on IBM PC's was not easy. You really had to want to run APL.
For example, we had to hack the equivalent of today's graphics card to replace the character ROM with one programmed to display APL characters. The more advanced modification allowed you to throw a mechanical switch and go between APL and non-APL characters.
Printing APL characters required using very specific printers and changing the print wheel or ball (IBM printers) with versions having APL characters.
Again, you really had to want to run APL to endure this. And, that, of course, affected adoption. You couldn't run APL on any random IBM PC or clone on a desk.
So, Iverson, despite identifying notation as one of the more powerful features of APL, ends-up "going commercial" by transliterating APL symbols into combinations of ASCII characters. Now you could run something that behaved like APL on any computer. And, of course, it was a complete abomination. Terrible thing.
I think I can say the APL community rejected J almost universally. Doing this was a terrible idea. And a short-sighted one at that. It wasn't long til all computers had the ability to display expanded character sets and eventually full graphical canvases where anything was possible.
J was a commercial reaction to a hardware problem that evaporated very quickly. J was not a commercial success for reasons obvious to those of us who used APL professionally every day for years: It was a hot mess and almost diametrically opposed to what made APL so incredible to use in solving problems through computing.
I would stay far away from J. As brilliant as Iverson had to be to create APL when he did, J, as far as I am concerned, was an almost unforgivable mistake.
I appreciate that Dyalog is free for non-commercial use, but I think that part of what makes a programming language successful is if people can start using it in small ways to make their work better. For instance, at my previous job we started using Go for small things and it eventually took on a larger role once we saw how well it worked out in practice. Had we needed to pay to use Go for those small things, we never would have even tried it in the first place. I realize that the Dyalog company needs to actually make money somehow, but I wonder if they are aware of this problem. I did email Dyalog to see if using Dyalog for my work at a non-profit constituted commercial use, but they said that it did. :(
Edit: The reason why I haven't used GNU APL is because it seems like getting it to work under Windows will be difficult, and my work is unfortunately a Windows environment. Maybe I should give it a shot anyway.
And I agree, I would pay, even if it's not that much. The commercial license comes with a too steep price tag. I'd rather opensource my code or contribute where I can.
Also, the <- instead of = in R comes from APL, although that's relatively minor. :)
[1] https://scottlocklin.wordpress.com/2013/07/28/ruins-of-forgo...
This was an amazing combination of languages to have in my toolbox simultaneously. The mental calisthenics alone were truly enjoyable to me.
As much as I love it I have to say one of the issues with APL is that it was way ahead of it's time. Because of that it struggled to run on computers of that era.
I was introduced to the language in 1982~ish. Like I said, I used it extensively, attended and presented at APL conferences and even have a picture of my younger and dorkier self with Ken Iverson (creator of APL).
This struggle to run on hardware that could not handle the huge expansions and contractions of memory utilization inherent in the way APL works multidimensional data sets like putty meant it could not compete where other languages had no issues at all. On PC's you were limited to 640K of RAM. It would be years before you could have enough memory to almost not care about utilization.
As the language didn't really catch the attention of the CS masses it failed to evolve as it could have over the years. And this is the reason I would suggest today it is not much more than a curiosity.
I'm sure there are some out there using it professionally. If I were to guess i'd say it must be mostly companies with a significant installed based of APL software they don't dare try to rewrite.
I firmly believe that the future of computing has to be in an APL-like language. By this I mean that we need to evolve to a notation for computing, rather than text for computing. Iverson himself wrote an excellent paper titled "Notation as a Tool for Thought".
The issue is that in order to evolve APL one has to have a deep understanding of APL. And, frankly, there are but a few of us around who have that understanding. Not sure how many of this group would embark on the mission to create a true next-generation APL worthy of it's lineage.
I read and I smile every time I see the language come up on HN. I taught my kids some APL and they are always blown away by it. Not sure where to go from here.