News for hackers. Stuff that matters.
Re:U.S. To Build Two Flagship Supercomputers for National Labs by rtkwe on Friday November 14, 2014 @03:27PM (#8609297)
The general reason comes down to inter-node bandwidth. Things like Google's search or SETI are able to use standard interconnect (or the open internet in SETI's case) because the problem, link counting and signal analysis, can be broken down into individual pieces that don't interact very much between sections of the computation.

Things like physics simulations, weather etc., have a lot of interaction between any divisions you could try to draw to be able to split the information between nodes. To work with these problems you need a faster interconnect between processors than ethernet provides, infiniband seems a popular choice but I'm no expert on the details of their architecture. These problems also need to move in relative lockstep meaning loosely connected systems don't work as well or at least don't provide any real advantages. Node homogeneity makes these tightly coupled processing nodes easier to manage.

Occasionally there will be upgrades to a super computer but generally they're run 24/7 until they're too far behind the technology, both in speed and lower power consumption, that it becomes cheaper to completely replace it than to try to do an upgrade that will replace more than half the cluster's total electronics to begin with.

Re:U.S. To Build Two Flagship Supercomputers for National Labs by epistasis on Friday November 14, 2014 @04:39PM (#8609650)
>The general reason comes down to inter-node bandwidth

And sometimes even more than bandwidth, inter-node latency, which is where Inifinband really shines over ethernet. Calculations which have high levels of node-to-node dependencies are pretty much the difference between super computers and the type of stuff that large internet companies compute on.

I hear (but don't personally know) that Google-style large data center installs are moving towards the CLOS-style networks that have been popular in HPC for a long time. These network topologies give equal bandwidth between any pair of nodes, as well as nearly equal latency.

Re:U.S. To Build Two Flagship Supercomputers for National Labs by mscman on Friday November 14, 2014 @05:29PM (#8609900)
Not just Google. Amazon's "Enhanced networking" uses a feature in 10GbE that's been used in smaller HPC clusters for around 5-10 years now. And MS Azure has InfiniBand backing their highest-tier instance types.

Many datacenters are adopting HPC technologies to reach the scale they need.