News for hackers. Stuff that matters.
Re:GPU-Oriented PCIe Expansion Cluster by angry_octet on Sunday February 26, 2017 @06:44PM (#13740727)
Depends on your bisection bandwidth requirement and software cost. If you want to do MPI it is going to be cheaper to buy some InfiniBand cards. But if you have a small cluster and tightly coupled code, or large I/O requirements, then a PCIe switch might be the go. These get used on some video on demand systems and signal processing architectures:

https://www.microsemi.com/products/drivers-interfaces-and-pc...

The Dolphin systems are tuned better for computation: http://www.dolphinics.com/products/IXS600.html

I expect small PCIe NVMe external storage systems to become quite common in the near future, because enterprise systems need multipath storage for reliability; 8GB/s FC is too slow for SSDs, let alone NVMe, same with SAS bus expanders.

https://events.linuxfoundation.org/sites/events/files/slides...

Re:GPU-Oriented PCIe Expansion Cluster by kierank on Monday February 27, 2017 @06:41PM (#13749687)
The Dolphin systems seem more expensive than just buying 4U servers.
Re:GPU-Oriented PCIe Expansion Cluster by angry_octet on Monday February 27, 2017 @09:03PM (#13750521)
You only buy them when you can't fit your problem into one server (i.e. not enough memory or I/O) or you need redundancy (multipath I/O, synchronised memory). In those cases they can be a lot quicker/cheaper/more reliable than trying to solve it with more computers, protocols and ethernet.