News for hackers. Stuff that matters.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by gnufx on Saturday July 28, 2018 @10:48AM (#17633382)
No, it isn't a supercomputer. These things (whether they be Parallela, Pi, or whatever) are generally at least an order of magnitude too low in every parameter relevant compared with a single node of a real HPC system.

A typical University system from that time might have been have been >~100 nodes with 16 Sandy Bridge cores, ~20 DGEMM GFLOPs and >=2GB ECC memory per core, a QDR Infiniband interconnect (~1μs latency, 40GB bandwidth), and >100TB of Lustre parallel filesystem. Also you could actually manage the hardware.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by felixgallo on Saturday July 28, 2018 @11:27AM (#17633541)
Don’t be like that. The shorthand here is that this is a novel, massively parallel architecture, much like those in supercomputers. If it had been adopted and scaled up, it could have had interesting applications.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by amelius on Sunday July 29, 2018 @06:40AM (#17637401)
But the parallelism isn't the hard part. The hard part is getting the communication structure right. Scaling up becomes difficult if the complexity of communication is e.g. quadratic in the number of nodes.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by supercomputery on Sunday July 29, 2018 @12:16PM (#17638830)
if scaling up were so easy, every country in the world would have made 1000s by now. there is a reason scaling up is hard, hard, hard problem. dont make it sound trivial how difficult building actual supercomputer architectures is.

its easy for anyone who doesnt know the challenges to say - yeah, we will just scale it up using duct tape and straws

Re:My Name is Brian and I Build Supercomputers in My Spare T... by mmt on Sunday July 29, 2018 @04:25PM (#17640081)
> The shorthand here is that this is a novel, massively parallel architecture

I'll bite.. I'm actually not at all experienced with supercomputers or HPC clusters, only 1990s proprietary MPP systems, what has been termed "grid computing" [1], and currently-popular systems like Hadoop.

In what way is this novel?

[1] e.g. distributed compilation, semiconductor simulation/test, or similarly embarassingly parallel workloads requiring little formal coordination beyond traditional batch scheduling resource allocation

Re:My Name is Brian and I Build Supercomputers in My Spare T... by gnufx on Sunday July 29, 2018 @09:14AM (#17637962)
"Like that" as in trying to educate? Too bad.

This described something isn't "massively parallel" and couldn't scale up. I'm happy with calling the big Sunway- and PEZY-based systems supercomputers and hope that Neo turns out well.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by yjftsjthsd-h on Sunday July 29, 2018 @07:00PM (#17640849)
"Like that" trying to tear down a fun little hobby project.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by ianai on Sunday July 29, 2018 @09:52AM (#17638117)
You’re violating the rules of HN.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by yjftsjthsd-h on Sunday July 29, 2018 @07:01PM (#17640858)
More precisely:

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

- https://news.ycombinator.com/newsguidelines.html

Re:My Name is Brian and I Build Supercomputers in My Spare T... by ianai on Sunday July 29, 2018 @11:49PM (#17641910)
I saw OP threw in technical points where the hobby machine is not like a production HPC box - after issuing a dismissal of the work entirely. HN does best when it gives a hobbyist a good faith appraisal of his/her work. A hobbyist is of course not going to have the resources to do HPC at home - usually.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by mmt on Monday July 30, 2018 @06:05AM (#17643341)
> I saw OP threw in technical points where the hobby machine is not like a production HPC box - after issuing a dismissal of the work entirely.

Which would make it a non-shallow dismissal, as it provided a number of specific discussion points, which are, in fact, being discussed. The guidelines don't, after all, discourage dismissal (even dismissal in entirety), only shallow dismissal.

> HN does best when it gives a hobbyist a good faith appraisal of his/her work.

This strikes me as a very subject assertion, all around. It's a matter of opinion on how HN does best, and it's a matter of opinion what constitutes a good faith appraisal, especially as it relates to a hobby.

That said, I might agree for a "Show HN" post, but not for a repost of something 4 years old, with the arguably clickbaity (and patently, unarguably controversial) unqualified term "supercomputers" in the title.

> A hobbyist is of course not going to have the resources to do HPC at home - usually.

If not, I'm left to wonder, what's the point? Where is the performance "high" or the computing "super"?

I could somewhat wrap my brain around a hobbyist building a very small (in node count) HPC cluster as a testbed. I could also understand if the goal were something like cost-performance efficiency improvement (for purchase and/or operating cost). Is that what's going on here? I didn't get that from the article, and only one comment claimed the architecture is "novel" without specifying in what way.

I didn't even see anything to suggest scalability/modularity that's easier than "real" HPC, but that could be because I missed it among the emphasis on handmade construction.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by mmt on Sunday July 29, 2018 @07:01AM (#17637459)
> a QDR Infiniband interconnect (~1μs latency, 40GB bandwidth)

This is the key aspect of actual HPC clusters that appears to be missing from the pretenders.

Some of the sibling comments point out that these scaled-down versions can be useful for testing in advance of deploying to (or building in the first place) the real cluster. However, that only makes sense if everything is scaled reasonably proportionally, and that can be tough with I/O, if latency and not just bandwidth, is important.

I can imagine a tiny platform like a raspberry pi being credibly interconnected with gigabit ethernet, but even a vaguely modern low-end x86 cpu can saturate it (and maybe even 10GE), never mind the (100x?) latency compared to IB.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by danieldk on Sunday July 29, 2018 @08:12AM (#17637725)
This is the key aspect of actual HPC clusters that appears to be missing from the pretenders.

That completely depends on your workload. For example, we did (natural language) parsing on thousands of CPUs on one of the large European computation grids. This work was largely CPU bound, the input and output data is relatively small, so the interconnect speed is not a big factor. The problem is embarrassingly parallel, so the primary consideration is having enough CPUs with a reasonable amount of memory.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by gnufx on Sunday July 29, 2018 @09:21AM (#17637981)
When we say "HPC" generally, we're talking about being able to run computation that isn't embarrassingly parallel. The bulk of work on typical general purpose systems tends to be DFT-type materials science and finite element/CFD engineering. Obviously if you have a special purpose system you can made different trades-off.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by mmt on Sunday July 29, 2018 @01:43PM (#17639329)
To be fair, you original comment did use both "supercomputer" and "HPC". I'd argue that, although in most people's minds, the two are indistinguishable (perhaps like the words "envious" and "jealous"), but carry a unique significance to practitioners.

That said, any lay definition of supercomputer would seem to require actual performance (e.g. high FLOPS rate), which trivally supports the initial assertion.

One can call something like the Folding@Home project a distributed supercomputer, but it's not that interesting, especially to build such a system, for anything embarrassingly parallel.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by danieldk on Monday July 30, 2018 @05:02AM (#17643093)
To many practitioners (people using HPC rather than people designing HPC), HPC means an aggregation of computing resources to deliver orders of magnitude of higher performance than one could get from a single workstation.

Obviously if you have a special purpose system you can made different trades-off.

It should be obvious, but the post I was reacting to was claiming that it cannot be 'real HPC' without high-speed interconnects. It depends on what you are a cluster/HPC facilities for. Typically institutions (such as universities) have different facilities with different trade-offs.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by mmt on Monday July 30, 2018 @05:43AM (#17643243)
> HPC means an aggregation of computing resources to deliver orders of magnitude of higher performance than one could get from a single workstation.

> the post I was reacting to was claiming that it cannot be 'real HPC' without high-speed interconnects

That was me, and that's still my point. The perception of users that have the embarassingly parallel workload isn't particularly meaningful. We already had the pretty generic terms "grid computing", "distributed computing" and "cluster".

What's the point of nomenclature like "HPC" if it describes something that's indistinguishable from that single workstation merely multiplied by some large number, somehow attached over the Internet?

If it's not "real HPC", then what is a better term of art to describe the sort of computing that can deliver high performance to workloads that include ones that are not embarrassingly parallel?

Re:My Name is Brian and I Build Supercomputers in My Spare T... by gnufx on Sunday July 29, 2018 @09:27AM (#17638012)
Exactly, on the scaling, but even then you probably won't see potential problems with, say, deadlock, that may hit you on a real system. You can't stress latency enough, as you'll now if you've ever seen something like cp2k limping over TCP/Ethernet.

By the way, if you want to simulate performance of real coes on real systems (as you should), you use programs like dimemas, simgrid, and others with real or projected parameters.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by danieldk on Sunday July 29, 2018 @01:45AM (#17636717)
Ok, a poor man's supercomputer then ;). I think such projects are great to experiment with parallel computing, get experience with ARM, and just plain fun.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by rbanffy on Sunday July 29, 2018 @02:39AM (#17636853)
IIRC, you can test your OpenMPI workloads on a desktop cluster before you push them to your really big machine.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by PaulKeeble on Sunday July 29, 2018 @04:13AM (#17637070)
Various super computer labs are using single board computer clusters of similar node size (but obvious less compute) in order to simulate the parallel compute issues without consuming the main cluster. It is just another step in the testing process for complex software that might very well waste cluster resources.
Re:My Name is Brian and I Build Supercomputers in My Spare T... by rbanffy on Sunday July 29, 2018 @02:37AM (#17636844)
It's not a number cruncher, but it emulates the same issues you see on an HPC cluster. To the point it's actually used for that.

https://www.extremetech.com/extreme/259553-750-raspberry-pi-...

Having said that, a cluster of generic boxes doesn't really deserve to be placed in the same category as machines engineered from the ground up to be a supercomputer.

Re:My Name is Brian and I Build Supercomputers in My Spare T... by patrickg_zill on Sunday July 29, 2018 @10:40AM (#17638357)
Is the infiniband 40Gbytes/s or 40Gbits/s? I seem to recall that it is 40 gbits/s?
Re:My Name is Brian and I Build Supercomputers in My Spare T... by mmt on Sunday July 29, 2018 @04:09PM (#17640011)
It would be 40 gigabits per second, and I hadn't even noticed the capital B (which I assume was a typo) until you pointed it out.

I was about to say that this is a consistent convention for networking to use bits instead of bytes, but I realized this breaks down with something like IB that has been used as a bus-like interconnect and a storage network/fabric (both areas where bytes are the more common convention in rates).

Even today, 40 gigabytes per second would be pushing the I/O capacity of a typical Xeon, though not a 128-lane Epyc, and I'm not sure about POWER9.

With 200Gb/s IB with both bandwidth and latency on the same OOM as other-socket RAM, it's also pretty close to pushing that capacity, which is, of course, the whole point, I suppose.