News for hackers. Stuff that matters.
Re:Putting GPUs to work with Kubernetes by tokenizerrr on Wednesday April 19, 2017 @06:42AM (#14146794)
So Ceph is the preferred storage provider? I've noticed there is a huge list, including GlusterFS. Do you have experience with any of the other ones?
Re:Putting GPUs to work with Kubernetes by ferrantim on Wednesday April 19, 2017 @12:31PM (#14149037)
Ceph is not a good solution at all for databases. Your performance is going to be terrible.

Ceph is object-based SDS solution designed to take servers with local drives and create a SAN out of them. In order to do this, they take each LUN (Ceph volume) and scatter the data across all nodes in the cluster. They do not assume that applications will run on these servers themselves... they assume compute is elsewhere, like a traditional SAN. The goal is to replace a SAN with servers, not create a converged platform. Also, Ceph was designed during an age where an Intel server did NOT have tier-1 capacity (8 - 20 TB), which is why they shard a volume across so many servers.

This causes a problem for modern applications like Cassandra, Mongo, Kafka etc, where they like to scaleout themselves and want a converged system, where data is not scattered, but on the node where an instance of that cluster runs. Ceph also disrupts (undo) the HA capabilities that these scaleout applications have (For example, a Cassandra instances data will not be on a node on which it thinks it is).

Re:Putting GPUs to work with Kubernetes by tokenizerrr on Wednesday April 19, 2017 @09:01PM (#14153163)
Do you happen to have any suggestions for alternatives to look into?
Re:Putting GPUs to work with Kubernetes by ferrantim on Monday April 24, 2017 @03:52AM (#14182581)
You could look at Portworx (disclosure, I work there so am biased, but you can test it yourself for free)
Re:Putting GPUs to work with Kubernetes by marcoceppi on Wednesday April 19, 2017 @08:37AM (#14147258)
Gluster is a good alternative, Ceph can be prickly if you don't want a block device and instead just need a filesystem. CephFS fills sort of the same role, but does so on top of Ceph rbd.

Since GlusterFS /can use/ NFSv4 as a client, it should work with the stuff @samco_23 uses

Re:Putting GPUs to work with Kubernetes by samnco on Wednesday April 19, 2017 @07:39AM (#14147008)
Ah you are right, I forgot about glusterfs. My bad.

Canonical at this stage only supports Ceph commercially, but it doesn't mean GlusterFS is not a good option. I haven't tried it myself, so can't tell.

Anyone?