News for hackers. Stuff that matters.
Re:Why favor PostgreSQL over MariaDB/MySQL? by fusiongyro on Tuesday September 05, 2017 @03:07AM (#15173081)
Advanced replication strategies in MySQL and Oracle mostly cause hair loss and divorce. Everywhere I have worked that uses these databases eventually convinces itself that it needs futuristic replication technology and then goes on to suffer from outages and weird corruption. It turns out, distributed consistency is a hard problem. Oracle and MySQL both benefit monetarily from people assuming it will work and then activating it, only to find out a month or two later that it's a complicated problem that requires actual planning and oversight.

I think Postgres will be a better product when it incorporates similar functionality, but the implementations in MySQL and Oracle lie somewhere on the border between false advertising and reckless endangerment. I appreciate that Postgres not having it as two-lines of configuration sends a message that you are making a commitment. Nearly everything else in Postgres is set-it-and-forget-it. You really want to have a DBA if you're going to try to make replication work. When you read the docs on having a hot standby and how much of a commitment that is and sense yourself getting tired, you're starting to appreciate what you might be getting yourself into with single-master replication, let alone multi-master.

Re:Why favor PostgreSQL over MariaDB/MySQL? by barrkel on Tuesday September 05, 2017 @03:23AM (#15173144)
This has not been my experience of multi-master asynchronous replication in MySQL at work. We use only one replica for actual work and the second master is read-only until failover - and is only write enabled after the primary is disabled - and it works quite well.

Getting replication bootstrapped synergises well with having a smooth backup restore procedure, similarly any hiccups in replication can be dealt with using restore.

We have other release QA procedures that use restored backups for testing, so our restore procedure is extremely well exercised. Really fast restores using the Percona tools is a considerable bonus of the MySQL world.

Re:Why favor PostgreSQL over MariaDB/MySQL? by kelnos on Tuesday September 05, 2017 @03:56AM (#15173292)
> This has not been my experience of multi-master asynchronous replication in MySQL at work. We use only one replica for actual work and the second master is read-only until failover

That's not multi-master. That's single master with a hot standby.

> any hiccups in replication can be dealt with using restore.

Restore isn't an option when your database is so large that it takes most of a day to copy the backup onto the replacement master to restore from it. (Not speaking hypothetically here.) Obviously restoring from a backup is still there as a last resort in this case, but you really don't want to have to go there.

Re:Why favor PostgreSQL over MariaDB/MySQL? by pas on Tuesday September 05, 2017 @05:20AM (#15173603)
Galera builds on that, and it works very well. It has weak points, but if you build a working Galera cluster and test it (test the failovers, test [re]bootstrapping a node), you can then be confident that it'll continue to work as intended for a long time.

There are rough edges. Upgrades, recovery from corrupted hot backup, etc. But it's very easy to rebuild the cluster in flight, for example when changing some fundamental cluster property, or after a PEBKAC, (with a not too big read-only window), though it requires plus one node, a regular MySQL/MariaDB server that acts as a donor (where Galera can bootstrap from), but that's not too surprising if you want to at least serve read-only requests.

Re:Why favor PostgreSQL over MariaDB/MySQL? by seanp2k2 on Tuesday September 05, 2017 @06:14AM (#15173833)
You also can't use Galera on RDS, which is a deal breaker for some: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_M...
Re:Why favor PostgreSQL over MariaDB/MySQL? by stephenr on Tuesday September 05, 2017 @07:41AM (#15174204)
Sounds like a reason not to use RDS more than a reason not to use Gallera.
Re:Why favor PostgreSQL over MariaDB/MySQL? by pas on Tuesday September 05, 2017 @11:15AM (#15176022)
Why would you use RDS? AWS is overpriced.

If you really want no-ops deployment, use something that requires minimal amount of ops, like ElasticSearch. Just give it an obscene amount of RAM and it'll work fine for ages.

RDS or not, if you want to run a big site, you'll need (dev)ops expertise. And a lot of it. Famously, Netflix - the big full-cloud business, - employs the most aggressive performance engineering and other "auxiliary" development groups. (Because AWS is fckn!1 expensive, and when you can run on a thousand less nodes, it'll cost you less. And yes, of course Netflix gets a big discount, but you won't, at least not initially.)

Re:Why favor PostgreSQL over MariaDB/MySQL? by manigandham on Tuesday September 05, 2017 @08:30PM (#15180217)
Neither Netflix nor any of the top "startups" are anywhere near aggressive performance. In fact they're usually full of slow fragile tech that only looks great to blog about.

HFT, adtech, scientific engineering and many other industries are magnitudes farther ahead.

Re:Why favor PostgreSQL over MariaDB/MySQL? by stephenr on Tuesday September 05, 2017 @12:28PM (#15176716)
> no-ops deployment

This is the worst joke in tech really. Developers/Managers all jumping on the "cloud" bandwagon because "hey we don't need ops" and then you end up with "Galera is such a POS, you can't use it with RDS".. <facepalm>

Re:Why favor PostgreSQL over MariaDB/MySQL? by otterley on Wednesday September 06, 2017 @01:53AM (#15181521)
> If you really want no-ops deployment, use something that requires minimal amount of ops, like ElasticSearch. Just give it an obscene amount of RAM and it'll work fine for ages.

I can assure you this is not the case if your indexes are very large and their number is constantly growing, and/or clients are doing very complex queries that stress the system.

Re:Why favor PostgreSQL over MariaDB/MySQL? by pas on Thursday September 07, 2017 @04:48PM (#15195570)
The same thing applies to anything really, if you try to "shard" (partition, or simply do multi tenancy) based on files, MySQL tables, MongoDB collections, and of course elasticsearch indexes.

And you have to plan ahead and start with a large number of shards, to be able to scale your ES cluster.

Yet it's much better than trying to get some kind of SQL based cluster going, because those require knowledge of a lot of subtle internal details.

Re:Why favor PostgreSQL over MariaDB/MySQL? by barrkel on Tuesday September 05, 2017 @09:05AM (#15174692)
It's multi-master from MySQL's settings perspective.

Restoring a backup takes minutes for multi-TB restores.

Re:Why favor PostgreSQL over MariaDB/MySQL? by IgorPartola on Tuesday September 05, 2017 @09:21AM (#15174799)
Just because MySQL calls it multi-master doesn't make it so. What you described is single master with a hot spare. Real multi master is when a client can write to both servers and they will be consistent by the magic fairy dust that is replication.
Re:Why favor PostgreSQL over MariaDB/MySQL? by kelnos on Tuesday September 05, 2017 @07:43PM (#15180007)
How exactly do you get that multi-TB backup onto the database server's disk so quickly? Even with a gigabit network interface, that's only 125MB/s, which is over two hours[0] to transfer 1TB. Even if you have a 10 gigabit network, that's ~13 minutes[0].

Also consider if the backups are compressed -- a 1TB compressed file also takes time to uncompress. If the data compresses well, you could even run into the disk being the bottleneck, not the network.

[0] Well, more than that; those numbers assume you're actually getting the link rate with no overhead, which of course isn't reality.

Re:Why favor PostgreSQL over MariaDB/MySQL? by beagle3 on Tuesday September 05, 2017 @02:26PM (#15177854)
Minutes for multi TB restores? Only if both your main and your backup are on SSD, which is still not common for multi TB databases, I think.

1TiB=1,000GiB; if your drives do 300MB/sec, that's still one hour.

Re:Why favor PostgreSQL over MariaDB/MySQL? by CaptainZapp on Tuesday September 05, 2017 @08:56AM (#15174635)
Interesting side note:

A significant number of Oracle shops use Sybase Replication Server to replicate from an Oracle - to an Oracle database.

Go figure...

Re:Why favor PostgreSQL over MariaDB/MySQL? by will_be_no_ai on Tuesday September 05, 2017 @01:26PM (#15177271)
RE: Oracle Replication - Have you looked at DbVisit? Compared to the PhD you need for GoldenGate, you can get by with a high school diploma with this stuff.

http://www.dbvisit.com/

Re:Why favor PostgreSQL over MariaDB/MySQL? by kaamos on Tuesday September 05, 2017 @03:12AM (#15173093)
> Nearly everything else in Postgres is set-it-and-forget-it.

There are many people who would disagree with that. Talking to Uber engineers would probably be a good idea to dispel that myth.

Re:Why favor PostgreSQL over MariaDB/MySQL? by glogla on Tuesday September 05, 2017 @05:11AM (#15173574)
Uber switched from MySQL used as relational database, to Postgres used as relational database, to MySQL used as key-value store.

Nobody is saying MySQL is a bad key-value store!

Re:Why favor PostgreSQL over MariaDB/MySQL? by pmarreck on Tuesday September 05, 2017 @09:49AM (#15175087)
Is key-value store the most stable scalable architecture? Reddit, I believe, uses the same idea
Re:Why favor PostgreSQL over MariaDB/MySQL? by fusiongyro on Tuesday September 05, 2017 @11:17AM (#15176037)
To restate your other reply, key-value allows you to sweep consistency problems under the rug. What happens if two transactions try to update the same row? One of the transactions fails. What happens if two clients write different values to the same key in a multi-master key-value store? The last one in wins and the first write is just lost.

Many of the NoSQL databases have provisions for these situations so they can handle both writes but bubble the decision up to the application layer. CouchDB, for instance, will tell you you've got multiple versions and has a way for you to tell it how to resolve the conflict. CRDTs let you bake the logic in for handling a conflict. MySQL does not do any of that. If your database consists of "likes" and other low-value data like that, you may not care if you're losing data at some low rate. Reddit is in that situation, and the decision seems to work for them. But I don't think it generalizes to all situations.

Re:Why favor PostgreSQL over MariaDB/MySQL? by softawre on Tuesday September 05, 2017 @10:38AM (#15175627)
If you give up ACID (the C) you can scale more (the P), sure.

https://en.wikipedia.org/wiki/CAP_theorem

Re:Why favor PostgreSQL over MariaDB/MySQL? by fusiongyro on Tuesday September 05, 2017 @03:18AM (#15173116)
I read their article, and a dozen of the rebuttals. If you want to use MySQL badly enough, you can.
Re:Why favor PostgreSQL over MariaDB/MySQL? by merb on Tuesday September 05, 2017 @05:29AM (#15173644)
> There are many people who would disagree with that. Talking to Uber engineers would probably be a good idea to dispel that myth.

not a good idea, somebody who let their database master run out of storage space is probably not a good source.

Re:Why favor PostgreSQL over MariaDB/MySQL? by user5994461 on Tuesday September 05, 2017 @06:00AM (#15173772)
Uber is a bad reference for good practices.
Re:Why favor PostgreSQL over MariaDB/MySQL? by jeltz on Tuesday September 05, 2017 @09:47AM (#15175048)
The Uber blog post got a lot of things wrong and if that is representative of their knowledge of PostgreSQL I would not suggest using them as an authority on PostgreSQL or databases in general.