News for hackers. Stuff that matters.
Re:Oracle built a 1,060 node Raspberry Pi cluster by djaque on Thursday November 21, 2019 @03:36PM (#21599297)
I think this is cool, but engineering for engineering's sake aside I've seriously looked at building a raspi cluster a couple of times and have always concluded that it isn't worth it.

The real selling point for me is energy efficiency and price, but the reason I could never make it happen is because most of the applications I want to use are x86 only and I don't have the resources to be a trailblazer in that area. In addition sourcing becomes an issue with raspberry pis when you aren't a big company like Oracle. It's hard to buy more than a handful as an individual.

Re:Oracle built a 1,060 node Raspberry Pi cluster by TwoNineFive on Thursday November 21, 2019 @09:27PM (#21601913)
This is purely for education and show.

I worked on two top-50 supercomputer clusters for science applications. You learn a lot by doing. You find out what happens to your electrical power, TFTP, DHCP, and syslog servers, and other infrastructure when those hundreds/thousands of nodes boot up all at once. Your Cisco switch reloads when you add a secondary IP address to all of your network interfaces because it's CAM table overflows. You learn how to automate without stuff like Ansible and Chef because they don't exist yet.

Re:Oracle built a 1,060 node Raspberry Pi cluster by intpx on Friday November 22, 2019 @09:19AM (#21605723)
Ha, it was my experience with HPC that lead me to convince our CCIE network admin that CAM tables even could overload on a vlan of that size (we were using layer 3 switching to route tons of other vlans and this particular vlan was very active in traversing those boundaries and was also using multicast and lots of other weird things). We were seeing an issue when where computers on the same switch would lose their ability to see each other, even if they could see other machines on a different switch and everything about it smelled like the cam table was not accepting new entries. I finally convinced him to clear it out on one affected port and it worked.
Re:Oracle built a 1,060 node Raspberry Pi cluster by willis936 on Thursday November 21, 2019 @04:00PM (#21599563)
In case people reading play with raspis and aren’t aware: you can reduce power consumption by several percent by disabling peripherals. This wiki has some useful information for those looking to make raspi clusters.

https://www.pidramble.com/wiki/benchmarks/power-consumption

Re:Oracle built a 1,060 node Raspberry Pi cluster by jacquesm on Thursday November 21, 2019 @03:50PM (#21599451)
The real selling point for me would be physical isolation. If you run all of them single-core you wouldn't have to worry at all about snooping using the various side channel attacks.
Re:Oracle built a 1,060 node Raspberry Pi cluster by Skunkleton on Thursday November 21, 2019 @04:24PM (#21599803)
Yes, but how many RPis do you need to match the workload of a single x86 core? How much does it cost to cluster and maintain these systems? IMO you would probably have a better story all around if you based your cluster on low end AMD chips, or perhaps even old refurbished systems.

That's not to say this isn't cool engineering for engineering's sake. Just that if you are considering practical applications for this, maybe think twice.

Re:Oracle built a 1,060 node Raspberry Pi cluster by jacquesm on Thursday November 21, 2019 @07:00PM (#21601137)
When you need that guarantee of isolation you are picking your solution for different reasons than when you need a particular workload. So there is no point comparing the workload if that wasn't the deciding factor in the first place.
Re:Oracle built a 1,060 node Raspberry Pi cluster by Skunkleton on Friday November 22, 2019 @04:53PM (#21610512)
My point is that you can probably get physical isolation for less money on x86.
Re:Oracle built a 1,060 node Raspberry Pi cluster by jacquesm on Saturday November 23, 2019 @10:21AM (#21614537)
You're going to have to explain a bit more on how you intend to achieve that economy. I'm not aware of a single SOC+peripherals based on x86 that would beat the Pi in price/performance, is there such a thing?
Re:Oracle built a 1,060 node Raspberry Pi cluster by mappu on Thursday November 21, 2019 @05:38PM (#21600512)
> most of the applications I want to use are x86 only

Really? What software in this space (headless server cluster) is still x86 only?

Re:Oracle built a 1,060 node Raspberry Pi cluster by djaque on Thursday November 21, 2019 @09:06PM (#21601826)
Unfortunately some proprietary scientific software is compiled for x86 only and we just get access to the binaries. I don't like it, but that's just how some fields work.
Re:Oracle built a 1,060 node Raspberry Pi cluster by giancarlostoro on Thursday November 21, 2019 @05:07PM (#21600242)
The Raspberry Pi Zero's are ridiculously affordable. For the cost of a regular Pi 3 you can get one thats wireless with a case with 3 covers for the case (1 for the Raspberry Pi camera). So maybe the Zero W's are more of what you might want. If you dont need them all to have wireless then you can just get the Zero's instead.
Re:Oracle built a 1,060 node Raspberry Pi cluster by djaque on Thursday November 21, 2019 @09:07PM (#21601831)
By sourcing, I wasn't arguing about their price. The problem is that it's hard to find anyone who will sell you more than a handful of them at low prices.

I haven't checked recently, but at least that's what I remember.

Re:Oracle built a 1,060 node Raspberry Pi cluster by charwalker on Thursday November 21, 2019 @06:33PM (#21600927)
I'd want a zero for projects. I have a 3B for messing around with and wish I already had PoE out so I could just set and forget a Pihole box or something. At the moment my Pihole instance is on an x86 Raspbian VM for simplicity but I think tonight or this weekend I'll set up the 3B do be a backup and point my router to it as a secondary DNs server. At least it'd be using the Pi vs it sitting in my drawer.
Re:Oracle built a 1,060 node Raspberry Pi cluster by imtringued on Friday November 22, 2019 @04:13AM (#21604058)
That's a pretty stupid reason. The real reason ist that most software doesn't run on a cluster at all and an 8 core Ryzen costs less than $170 and $340 for a full system without a GPU but is significantly faster than a 10 node rasperry pi cluster ( doesnt even include sd card power supply network gear)