Of course it's not fair to compare this to an infiniband cluster (that's not the point of this exercise), but I'd really be interested to see a cluster built on $0.50 ARM chips with at least a gigabit ethernet interconnect. A couple of years from now -- given the low entry cost and lower infrastructure costs (cooling/power consumption/etc) -- that could be a game changer.
Dell's "Project Copper" - http://content.dell.com/us/en/enterprise/d/campaigns/project...
Boston Viridis - http://www.boston.co.uk/solutions/viridis/default.aspx
[0] http://newsroom.intel.com/community/intel_newsroom/blog/2012...
[1] http://www.hpcwire.com/hpcwire/2012-09-10/intel_weaves_strat...
One of my favorite systems questions is to have someone walk through the design and implementation of a system where all the machines in the system respond to a query Q based on the contents of a linked list L. The system has an API which consists of L <- M(op) (mutate list), R <- Q(id) respond to a query based on the contents of the list, and R <- S() report on the stability of the list. Start with M(op) being idempotent, then non-idempotent, Etc. Folks who've had a good introduction to state machines will immediately recognize and a number of problems that arise as you control correctness. If folks get through the whole sequence we're taking about a function f(C) which takes a correctness coefficent from 1.0 (fully correct) to 0.0 (unspecified) and look at the performance of the system across that range.
That kind of stuff you could easily do on a 48 node Pi Cluster.
I implemented neural network feature creation for a backgammon agent ("Automated feature selection to maximize learning in artificial intelligence").
Nowadays, I mainly do parallel machine learning on machines with higher network latency. I haven't used MPI since.
With the sort of work I do nowadays (large-scale ML and NLP), I generally need very little synchronization, i.e. my tasks are usually embarrassingly parallel. I typically save final results in a centralized store (DB or NFS) and look at it there.
I also use Hadoop where appropriate.
http://www.southampton.ac.uk/~sjc/raspberrypi/pi_supercomput...
As you can see from the pictures each of the devices uses its own power supply, so there should be quite some possibility to improve overall power consumption.
Anybody with more experience of using MPI ?