g0del_was_wr0ng

Well fair enough but that method of counting is not what everyone else does or assumes, so somebody just reading the title "120k writes per second" gets the wrong impression of whats going on.

(An uninitiated reader would assume you're comitting 120k rows per second from the title, whereas it's "only" 12k rows and "only" 3k actual operations over the wire. Still, 3-12k is pretty impressive)

By that logic a write to a postgres table would also count as N+1 writes, where N is the number of secondary indexes defined on that table?

And also, by the same logic, replicating a write to three machines counts as three writes when the replication factor is user-defined, right?

It's doing 12k rows in 3k user-issued write operations/transaction per second.

Counting any kind of "internal write effects" that result from a user write (i.e. write amplification) is obviously done to mislead in the benchmark and does not make it comparable to key-value stores.

12k writes/s is the number of rows that are written from a user perspective. So 12k/s is also the number you have to use when comparing it to key value stores. But of course, comparing Fauna with eventually consistent systems is not a really fair comparison. You don't make it fairer by misleading in your benchmark though.

Also, just because some other vendor posted a misleading benchmark on hn (I don't know if they did) that doesn't make it right or means you should do it. Just call them out on it too.

Well, you're missing that in Faunas case the writes are durably stored on N machines. I.e. their system provides fault tolerance in case a machine fails. You can't really do the same thing with postgres (without trading off full ACID compliance).

Yeah it's a bit more complex than that... The disk "IOPS" number on the box doesn't translate 1:1 or even linearly to number of committed durable transactions per second. You should try this with postgres and see how it goes.

Including your 9x write amplification in the number of "consistent writes" doesn't count -- like at all. I'm amazed nobody called you out on this yet.

You're doing 3k batches per second with 4 logical writes each, right? So that is at most 3-12k writes per second using the way that every other distributed database benchmark and paper counts.

Or otherwise - if you continue counting writes in this special/misleading way - you'd have to multiply every other distributed db benchmark's performance numbers with a factor of 3-15x to get an apples-to-apples comparison.

The 12k batched writes/sek through what I assume is a paxos variant is still pretty impressive though! Good to get more competition/alternatives for zookeeper & friends!

I agree that your PG numbers don't sound likely/factual -- the reason for your confusion is probably that somebody gave you untrue performance numbers for postgres or you're not comparing the same things. Is the 1m+ TPS something you measured yourself or "heard from a friend"?

If you ran the benchmark yourself, how did you achieve 1m durable writes/sec on a postgres machine/instance? [It's quite an achievement] On what kind of crazy hardware? How large was each write/row? Did you use the postgres network protocol to perform the writes?

I think y0ghur7_xxx was talking about a usecase where you want to talk to the backend datastore from an application running outside of your datacenter in an untrusted environment (like an iOS/Android app, or a JS web frontend).

In this case, one can make the argument that a custom proxy layer, running in your DC (that proxies between the database and your actual frontend app) should not be necessary if the database offers sufficient per-connection ACLs and is secure.

That's a big if though.

>> The only way you can scale is to adopt a query-free architecture

This is not really the case. There are database systems that can handle large scale and complex queries. Allthough usually at the price of providing reduced consistency guarantees.

So what do you do when the master for one of your shards fails? Do you drop all incoming writes for the shard on the floor? Or do you fail over to the shard's slave and promote the slave to the new master?

Since you said you don't have "auto healing HA" I assume you don't fail over, but discard/deny incoming writes until the master comes back up.

This is a valid approach, but, I don't see how it contradicts what I said at all:

  - I said you can't get full ACID and HA failover at the same time with postgres
  
  - Your scheme does not provide HA failover

I explicitly said that if you can forgo either full ACID or HA in case of a failure, postgres is fine.

No, but when your requirement is to "never loose a write regardless of which server fails" you have to make sure the mutation was acknowledged by more than one server.

So the least you have to do is to wait for a second server to confirm the write. But now what do you do when the network link between the two servers goes down in-between a write? The write might have already been applied on the first server or it might not have. The same goes for the second server. But you can't be sure either way.

For our thought experiment, let's assume that while the servers can't talk to each other anymore, some clients can still talk to each of the servers.

At this point, you either have to give up availability (i.e. don't accept any further writes until the fault is resolved) or consistency (i.e. potentially return unacknowledged/stale data from one of the nodes until the fault is resolved).

While returning unacknowledged/stale data is acceptable for some usecases, it does break ACID semantics. You can't get both ACID and "always-on" availability with a simple replication scheme like this.

If you want linearizability and HA, you need at least three servers and a more complex, quorum-based scheme. However, Postgres doesn't support that (AFAIK) - you have to use something like Zookeeper or CockroachDB. I think this is what zzzcpan meant.

(I realize that you only spoke about never loosing a write and didn't say anything about updates/queries. If you don't care about ACID, postgres with synchronous replication is a good highly available solution)