Re:China Tops U.S. in Supercomputers
by dekhn on Monday June 20, 2016 @03:21PM
(#11940585)
In many cases, problems partition nicely across machines without communication. Take molecular dynamics as an example of an "interesting" problem. One approach is MPI-based parallelism: partition the particles or other entities in the system across specific processors, compute all the individual forces, sum them up, and send the results to the processors that need it. This allows you a number of advantages: you can work with larger systems (ones that wouldn't fit in RAM on a single machine), you can speed up the total throughput, etc). But, it comes at a huge cost to developer productivity, and the machines have to have very low latency, high through primitives, so that all the processors are evenly balanced (Amdahl's law places upper bound on total throughput).
On the other hand, you can just run N independent simulations, and pool the statistics. In many cases, this is as good as if not better than running one long MD simulation. Here, all the processors share a common input file, but it's a single file, it only has to be communicated once. Any results from the processors are compacted, and sent to a large file server, which in itself is designed with partitioning since all the output files are independent. This approach won't let you work with large systems (unless your individual servers have more RAM), but it scales linearly to far more processors than tightly coupled parallelism, for far cheaper, and often, the scientific results are as good or better.
This shows up in many cases- in my 20+ year career programming, nearly all the problems I saw people run on supercomputers eventually could be run on CPUs with easier code and better performance. There are some irreducible codes for which only supercomputers are capable of meeting the required specs, and those are the only codes which should run on supercomputers. Otherwise you're pissing money on interconnect and hard to program environments).
In short: if you can convert your supercomputer job into a cloud cluster, you should do so, because it will save time and money. If you can't, then you should use a supercomputer.
Note also that cloud interconnects have gotten significantly better over the years- for me, 100Gbit throughput and latency (as long as the cluster has the total parallel bisection bandwidth) is fine for me and most other people.
On the other hand, you can just run N independent simulations, and pool the statistics. In many cases, this is as good as if not better than running one long MD simulation. Here, all the processors share a common input file, but it's a single file, it only has to be communicated once. Any results from the processors are compacted, and sent to a large file server, which in itself is designed with partitioning since all the output files are independent. This approach won't let you work with large systems (unless your individual servers have more RAM), but it scales linearly to far more processors than tightly coupled parallelism, for far cheaper, and often, the scientific results are as good or better.
This shows up in many cases- in my 20+ year career programming, nearly all the problems I saw people run on supercomputers eventually could be run on CPUs with easier code and better performance. There are some irreducible codes for which only supercomputers are capable of meeting the required specs, and those are the only codes which should run on supercomputers. Otherwise you're pissing money on interconnect and hard to program environments).
In short: if you can convert your supercomputer job into a cloud cluster, you should do so, because it will save time and money. If you can't, then you should use a supercomputer.
Note also that cloud interconnects have gotten significantly better over the years- for me, 100Gbit throughput and latency (as long as the cluster has the total parallel bisection bandwidth) is fine for me and most other people.