News for hackers. Stuff that matters.
Re:U.S. To Build Two Flagship Supercomputers for National Labs by Someone1234 on Friday November 14, 2014 @02:58PM (#8609154)
Ignorant question: Why are supercomputers still popular? I would have thought that their examples could equally be accomplished with a more much flexible array of less powerful nodes (see Google's search engine as an example).

That way instead of doing a "big bang" upgrade such as this, you just upgrade individual nodes as the technology allows and are almost always "current."

PS - I find their usage of the term "energy independence" hilarious. That was a term coined largely to justify fracking and other environmentally damaging practices in the US. I'm glad to see it has been over-used so much now third parties are using it to justify other projects...

Re:U.S. To Build Two Flagship Supercomputers for National Labs by rtkwe on Friday November 14, 2014 @03:27PM (#8609297)
The general reason comes down to inter-node bandwidth. Things like Google's search or SETI are able to use standard interconnect (or the open internet in SETI's case) because the problem, link counting and signal analysis, can be broken down into individual pieces that don't interact very much between sections of the computation.

Things like physics simulations, weather etc., have a lot of interaction between any divisions you could try to draw to be able to split the information between nodes. To work with these problems you need a faster interconnect between processors than ethernet provides, infiniband seems a popular choice but I'm no expert on the details of their architecture. These problems also need to move in relative lockstep meaning loosely connected systems don't work as well or at least don't provide any real advantages. Node homogeneity makes these tightly coupled processing nodes easier to manage.

Occasionally there will be upgrades to a super computer but generally they're run 24/7 until they're too far behind the technology, both in speed and lower power consumption, that it becomes cheaper to completely replace it than to try to do an upgrade that will replace more than half the cluster's total electronics to begin with.

Re:U.S. To Build Two Flagship Supercomputers for National Labs by epistasis on Friday November 14, 2014 @04:39PM (#8609650)
>The general reason comes down to inter-node bandwidth

And sometimes even more than bandwidth, inter-node latency, which is where Inifinband really shines over ethernet. Calculations which have high levels of node-to-node dependencies are pretty much the difference between super computers and the type of stuff that large internet companies compute on.

I hear (but don't personally know) that Google-style large data center installs are moving towards the CLOS-style networks that have been popular in HPC for a long time. These network topologies give equal bandwidth between any pair of nodes, as well as nearly equal latency.

Re:U.S. To Build Two Flagship Supercomputers for National Labs by mscman on Friday November 14, 2014 @05:29PM (#8609900)
Not just Google. Amazon's "Enhanced networking" uses a feature in 10GbE that's been used in smaller HPC clusters for around 5-10 years now. And MS Azure has InfiniBand backing their highest-tier instance types.

Many datacenters are adopting HPC technologies to reach the scale they need.

Re:U.S. To Build Two Flagship Supercomputers for National Labs by cjslep on Friday November 14, 2014 @03:22PM (#8609279)
> Why are supercomputers still popular?

Scaling.

A very easy out for me to explain is imagine if you could only simulate two molecules in a Molecular Dynamics simulation and both had to have 100 atoms or less. There's plenty of opportunity, sure. But what if you wanted to simulate an entire strand of RNA interacting with a protein? There are techniques and stable approximations to avoid the O(n^2) complexity of a molecular dynamics simulation but the next-generation hardware allows you to run the same simulations with the same limitations at a much bigger scale, or the previous-sized problem without as many assumptions. edit: There are other considerations of course; I am vastly oversimplifying with this example.

Also, the more simulations you can run the more uncertainties you can account for in Monte Carlo computations, so models can evolve to contain more details and less assumptions.

adding response to: > That way instead of doing a "big bang" upgrade such as this, you just upgrade individual nodes as the technology allows and are almost always "current."

The problem is that programming targeting a supercomputer usually involves taking advantage of specific features of the hardware to get the bleeding edge performance. Mixing and matching hardware quickly makes deployment of the code difficult in addition to writing the code to reliably squeeze every ounce out of every chip.

Re:U.S. To Build Two Flagship Supercomputers for National Labs by baltcode on Friday November 14, 2014 @03:05PM (#8609199)
I think in most cases supercomputers are in fact "flexible arrays of less powerful nodes". It's just that you need to change the architecture as you scale up the number nodes and type of nodes. Basically, you need fast data transfer and memory sharing between these nodes if you want to do anything but the most embarrassingly parallel problems.
Re:U.S. To Build Two Flagship Supercomputers for National Labs by Retric on Friday November 14, 2014 @05:22PM (#8609860)
Picture a hurricane in a movie. It's created by an artist and 'rendered' using lot's of processing power. However, each rendered frame is only dependent on the artist so you could render it backwards or farm out each frame to a separate computer.

Now, suppose your a scientist who want's to simulate a hurricane. You create a model for the first step (second, minute, or nanosecond) and can send that to a computer, but what do you send to the next computer. It needs results from the first simulation to run, but in that case you might as well just keep running the simulation on the same computer. Clearly to use a cluster your going to need to break up your simulation into pieces and send each part to a separate computer. But, now after each step all the nodes need to talk to all the other nodes to keep the simulation in sync. Worse the more chunks you separate the problem into the more parts need to be talking to other parts.

This is also why supercomputers tend to have equal strength nodes. Having one node finish before the other is pointless as it needs results from other nodes to continue. In the end you need not just massive bandwidth but also low latency. If a step takes 1/1000th of a second to run but then needs to wait a full second to run the next step then your simulation get's 1 step per second not per 1/1000th of a second.

You also want to back up each node so if one of them crashes you don't need to re run the simulation from scratch.

Re:U.S. To Build Two Flagship Supercomputers for National Labs by hvs on Friday November 14, 2014 @03:24PM (#8609285)
In a sense, these are essentially arrays of nodes, they just happen to be Tesla GPUs (which you can think of as video cards that don't have a video out). The advantage of these over individual machines is that the I/O speed between machines is much lower than between CPU/GPU.
Re:U.S. To Build Two Flagship Supercomputers for National Labs by Jtsummers on Friday November 14, 2014 @03:32PM (#8609321)
From the video in one of the links it seems that it's still nodes containing both CPU (POWER) and GPU (Tesla) (some number of each). But instead of PCI-e, it uses a new high-speed bus (NVLink) which allows for direct GPU-GPU communication and for the GPU and CPU to share memory at the same speeds (doesn't rely on PCI-e to connect GPU to CPU).
Re:U.S. To Build Two Flagship Supercomputers for National Labs by pmalynin on Friday November 14, 2014 @04:32PM (#8609621)
There is now also an RDMA protocol that understands GPU RAM