RethinkDB's niche is allowing a service to listen for changes to a dataset. As in, your Node application could listen for published events by Rethink, rather than your Node application long-polling the dataset for changes. "Real Time" data analytics can then be performed.

I've never done it, but I think it's possible to do this in Postgres as well using LISTEN and NOTIFY. You may want to do all of your database mutations through functions that ultimately call NOTIFY for this to work though. But uh, that's how you'd do it in postgraphql or postgrest anyway.

https://github.com/calebmer/postgraphql

http://postgrest.com/

You can definitely use LISTEN and NOTIFY to watch for changes in Postgres. Here's a presentation I did on that last year, though oriented more towards web apps than analytics. https://www.youtube.com/watch?v=PsorlkAF83s

For analytics I would want stronger guarantees than LISTEN and NOTIFY alone can provide. I explore options for that here: http://btubbs.com/streaming-updates-from-postgres.html

Its other niche is easy to deploy replication and sharding that actually works, actually scales, and does not lose data.

I've never seen that in any db I've worked with. If that's true, that's pretty huge.

It is true. It was NOT true a year ago, but it is now. I pushed it hard, Aphyr pushed it hard, and the RethinkDB team worked very hard addressing every issue we found. That's why RethinkDB failing is so tragic.

Damn, I'm simultaneously excited and sad now. I hope it survives as open source so I can use it for my next project.

Yeah it feels like they died right after getting everything right.

Firebase works like that. The polling of databases is a really obvious limitation of existing databases once you see the alternative.

(I work at Firebase)

Well it's easy to shard when you don't support joins or foreign key constraints.

Btw, as a person who works on Firebase, perhaps you can answer this question that has been bothering me about it. There's no way to model a foreign key relationship that's traversable in both ways without trusting the client application to update both directions, right? Isn't that a bit of a dealbreaker for anything other than a toy app? I would assume if you wanted to make something that's actually reliable, you'd have to run all your mutations through your own personal server instead of letting clients mutate Firebase directly, right?

You can use it as the mobile device connector as clientside auth, push, latency compensation, presense and offline persistence are big features on their own. For sensitive joins, have a server side joiner with administrative privileges. Still, it's work that should be easier I agree. Firebase's strength is not what other DBs can do, but what they cannot.

I'm curious what Firebase users do in real life. Do people realize this is a problem?

I can imagine making an app like that where all mutations go through my server and reads go through Firebase. How would this interact with the offline persistence feature though? I suppose you could allow the client to do some simple mutations like editing text fields. For the mutations that must go through the server, though, perhaps you could do optimistic local updates?

The latency compensation through the sdk is optimistic writes. If you shortcut to server, you have to reimplement that so that's not a good solution for many cases. It would confuse the offline persistence too.

Instead, your server should listen to the Firebase, rather than have writes go a separate path (which loses lots of useful features). I think the term now is a client-database-server architecture.

it's not actually a problem for 90% of apps. It's usually in the users best interest not to break their relationships in the db. Think chat. Think info services like Pokemon. Cross table transactions are relative minority in the space of useful apps that's can make money. For those more complex domains, you need technical know how, and Firebase is a useful system component in a larger architecture.

As soon as you want to view something through a different direction you run into this problem. Sure, you can embed chat messages into a conversation, but what if you want to look up every message by a particular user?

I would be surprised to find that 90% of apps don't have this kind of requirement. Pokemon is easy because that data is static.

Viewing all posts by a user is solved with classic denormalization. You maintain a separate index of post by user and do two writes per chat msg. It's a case where a user has no incentive not do the second write so it be fine to implement clientside. If however it was business critical, you would have a server side process listen to all chat messages and write the user to message index with the admin account. That index would essentially be eventually consistent.

[edit: this is an irrelevant remark] RethinkDB supports joins: https://www.rethinkdb.com/docs/table-joins/ "Like many traditional database systems, RethinkDB supports JOIN commands to combine data from multiple tables. In RethinkDB joins are automatically distributed—a join command is automatically sent to the appropriate nodes across the cluster, the relevant data is combined, and the final result is presented to the user."

Rethink joins aren't consistent (in fact, no Rethink multikey transaction guarantees any degree of consistency), so even though the original conversation was about Firebase IMO it is just as applicable to Rethink.

I'm talking about Firebase :P.

I searched and searched, but I couldn't find a download link on Firebase.com. Help?

I think Firebase is a closed source program that can only be used via "the cloud", i.e., via a web application. It's not something you can run yourself. In contrast, RethinkDB+Horizon would have soon provided similar functionality to Firebase, but with the option of self hosting.

Has Firebase been through Aphyr's Jepsen tests?

Seeing as how Aphyr can't deploy his own instances of Firebase to simulate network partitions, I don't think it has.

Pretty much anybody can appreciate that, right? Not just the "deep thinking people"?

Concur. We've been running a small cluster for about a year and we've had the same experience thus far.

isnt that what Realm is trying to do on the mobile side ?