For the performance piece, I'm looking at the attached link and it seems like neither read nor write throughput goes up as you add nodes. So will this allow you to scale your read / write volume horizontally or is it just for high availability? It doesn't seem like you can just add more nodes to get more write throughput. I'm guessing you'd still need to take advantage of read only replicas to scale read volume in addition to a cluster.
Also, since each copy needs a complete set of the data it does not seem like this clustering solution addresses growing data size. Correct?
Read throughput will go up by being able to distribute reads amongst the cluster. Write throughput shouldn't really get better; since all nodes have a copy of the data. Thus; the maximum node count is 9.
On the last question, there is data size, and there is working set size (what needs to be in memory). You can actually stretch working set size by sending certain queries (i.e. reports) to one of the nodes and keep the others for more transactional queries, but for storage on disk - yes, it is a multiple of however many nodes you have.
But also consider: Much of the pain I've had in large DBs is not being able to keep enough backups on fast media for quick restore (i.e. I'd like to have every day for the last 2+ weeks). From that pain point though, it's not a multiple - you just need to pick one node to backup.
The Tl;dr I give most people is that the Group Replication technology (innodb cluster) uses Paxos for group communication (with majority) and Galera uses Totem Single-ring Ordering (all nodes).
Eventually it stabilizes but I still have binary logs disabled because of an open issue over a year old.
I'm afraid that MySQL will have the same issues now that it's new.
Have you run Aphyr's Jepsen tests?
QA has definitely been one of the harder parts of creating a distributed MySQL. I don't work closely enough on that part to answer this question. Sorry!
To expand on your question a little - the MySQL router currently works better for single primary (it supports multi-primary but does not prevent stale reads). If you go third party, ProxySQL also supports Group Replication natively with multi-primary:
http://lefred.be/content/mysql-group-replication-native-supp...
I was chatting to the author about this at FOSDEM - since it is possible to keep track of nodes state by following GTIDs executed in the binary log stream. I believe this is how ProxySQL is doing it, but it's possible the final implementation differs :-)
Let me start of by describing how the clustering works:
- A cluster is 3+ nodes in a group, each with a complete set of data
- When you say 'COMMIT' the server you are connected to certifies your write-set with its peers (i.e. has anyone else tried to modify the same data). Once the majority agree, the transaction is considered successful.
- The distributing and certification of the transaction is synchronous, but the applying of the transaction is asynchronous. Thus you actually retain pretty good performance if the network latency is reasonable.
The performance question comes up a bit, so here's another post for more context:
http://mysqlhighavailability.com/an-overview-of-the-group-re...