News for hackers. Stuff that matters.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by paulsutter on Friday October 07, 2016 @04:46PM (#12663790)
To understand what happened to RethinkDB, you want to contrast it with MongoDB.

RethinkDB is amazing software and a real breakthrough. And the audience of deep thinking people who can really leverage the event model is ... limited. Today.

MongoDB is no breakthrough, but there are legions of people who just want to plug something together with PHP or Node. Given what they're doing, it works fine.

I sincerely hope Rethink can turn into a vibrant open source project and I'm willing to donate money to that. Given sufficient time, the world will learn to use it. And until then, it will grant superpowers to people willing to invest a little effort to understand and leverage it.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by peruvian on Friday October 07, 2016 @04:57PM (#12663848)
FWIW I found Rethink easier than Mongo, especially the query language. ReQL is actual JavaScript—Mongo's query language always seemed weird to me.

Mongo was "there" first and generated a lot of hype through sales and marketing. For a while and probably still now, a lot of Hackathons at college campuses are sponsored by Mongo and you can usually get a prize for using Mongo in your project.

Rethink never marketed themselves too much—it was one of those things you found out about in HN or from another dev.

I'm clueless about business stuff so I don't mean to say Rethink should've marketed heavily like Mongo did, but that's definitely a factor. I also think Rethink should've done Horizon earlier and offered paid hosting/deployment for it and Rethink users.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by chipotle_coyote on Friday October 07, 2016 @08:27PM (#12664930)
I admit I don't have a lot of experience with MongoDB, but RethinkDB was really easy to plug into Node. While we didn't support PHP officially, the PHP RethinkDB driver was maintained by RethinkDB's engineering manager, Daniel Mewes, and it looked pretty solid.

When I started at RethinkDB, it was before changefeeds were a thing. I always thought the "killer feature" was, well, ReQL. It had the qualities of an ORM/DB Builder in your native language of choice. There was a poster on the wall that said "Databases should be indistinguishable from magic"; I always thought that was a perfect pitch to the RethinkDB audience. I understand why they canted toward pitching the realtime aspects first and foremost after changefeeds came about, but I thought it had great power even if you weren't focused on realtime/event-driven aspects, and perhaps that should have been emphasized more.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by dkersten on Friday October 07, 2016 @08:34PM (#12664952)
In my opinion rethinkdb is super easy to get going and use. The low touch maintenance is what drew me originally.

However, learning to live without transactions takes some rethinking of the approaches taken. It's totally doable but it may not come easy if you're used to transactions in an RDBMS.

Of course in today's distributed multi service multi database world, learning to live without transactions is becoming more and more important anyway.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by lucio on Friday October 07, 2016 @08:51PM (#12665029)
if "Databases should be indistinguishable from magic" then somehow you should be able to get transactions
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by dkersten on Sunday October 09, 2016 @06:49AM (#12670691)
You can get transactions, but at a cost. Distributed transactions exist, they're just slow (and network-chatty). RethinkDB decided not to go that direction, I guess.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by a-no-n on Friday October 07, 2016 @06:05PM (#12664270)
Disclaimer: I have a bro crush on Ben Horowitz.

Exactly, they gave it a good go as a company but their burn rate was likely too high/too soon and differentiation insufficient (whether lack of actual novel features and/or communicating advantages in marketing). For example, a polar opposite of something like Couchbase which is used by giant enterprise shops to move away from Memcache and/or CouchDB.

It was also competing in a way Peter Thiel and Paul Graham warn about. Reinventing something without a vital quantum of improvement is like trying to juggle while climbing Half Dome.

I hope Rethink lives on as open source and adds MongoDB support. If they can hustle to encourage more people to try it by selling its obvious advantages, they have a slim but nonzero chance of rebooting.

Next time, gotta find more paying, alpha customers first and then start building. Then, try VC only after being capital-constrained when hiring/expansion/development is a blocker to realizing imminent profit. New category brands and segment-optimized products are riskier but more likely to be able to set up moats with defensible, competitive value.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by gogopuppygogo on Saturday October 08, 2016 @06:58AM (#12666484)
You nailed it with this "try VC only after being capital-constrained when hiring/expansion/development is a blocker to realizing imminent profit".

Don't raise money for an idea, raise money for a business that is capital constrained. VC is only one kind of funding you can try to grow with.

The best source of funding is revenue.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by thesmallestcat on Friday October 07, 2016 @05:00PM (#12663865)
For those not "in the know," what capability is that? Everything I hear implies a focus on ease of use: Easy replication/distribution, JS API, single number type (float64), usefulness when teaching novices, etc.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by alexbanks on Friday October 07, 2016 @05:23PM (#12664015)
RethinkDB's niche is allowing a service to listen for changes to a dataset. As in, your Node application could listen for published events by Rethink, rather than your Node application long-polling the dataset for changes. "Real Time" data analytics can then be performed.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tiglionabbit on Friday October 07, 2016 @05:59PM (#12664234)
I've never done it, but I think it's possible to do this in Postgres as well using LISTEN and NOTIFY. You may want to do all of your database mutations through functions that ultimately call NOTIFY for this to work though. But uh, that's how you'd do it in postgraphql or postgrest anyway.

https://github.com/calebmer/postgraphql

http://postgrest.com/

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by bjt on Friday October 07, 2016 @06:27PM (#12664400)
You can definitely use LISTEN and NOTIFY to watch for changes in Postgres. Here's a presentation I did on that last year, though oriented more towards web apps than analytics. https://www.youtube.com/watch?v=PsorlkAF83s

For analytics I would want stronger guarantees than LISTEN and NOTIFY alone can provide. I explore options for that here: http://btubbs.com/streaming-updates-from-postgres.html

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by api on Friday October 07, 2016 @05:28PM (#12664043)
Its other niche is easy to deploy replication and sharding that actually works, actually scales, and does not lose data.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by ericd on Friday October 07, 2016 @05:38PM (#12664109)
I've never seen that in any db I've worked with. If that's true, that's pretty huge.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by williamstein on Friday October 07, 2016 @05:49PM (#12664171)
It is true. It was NOT true a year ago, but it is now. I pushed it hard, Aphyr pushed it hard, and the RethinkDB team worked very hard addressing every issue we found. That's why RethinkDB failing is so tragic.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by ericd on Saturday October 08, 2016 @12:11PM (#12667431)
Damn, I'm simultaneously excited and sad now. I hope it survives as open source so I can use it for my next project.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by api on Saturday October 08, 2016 @11:19AM (#12667221)
Yeah it feels like they died right after getting everything right.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tlarkworthy on Friday October 07, 2016 @05:58PM (#12664224)
Firebase works like that. The polling of databases is a really obvious limitation of existing databases once you see the alternative.

(I work at Firebase)

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tiglionabbit on Friday October 07, 2016 @09:28PM (#12665168)
Well it's easy to shard when you don't support joins or foreign key constraints.

Btw, as a person who works on Firebase, perhaps you can answer this question that has been bothering me about it. There's no way to model a foreign key relationship that's traversable in both ways without trusting the client application to update both directions, right? Isn't that a bit of a dealbreaker for anything other than a toy app? I would assume if you wanted to make something that's actually reliable, you'd have to run all your mutations through your own personal server instead of letting clients mutate Firebase directly, right?

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tlarkworthy on Saturday October 08, 2016 @02:39AM (#12665950)
You can use it as the mobile device connector as clientside auth, push, latency compensation, presense and offline persistence are big features on their own. For sensitive joins, have a server side joiner with administrative privileges. Still, it's work that should be easier I agree. Firebase's strength is not what other DBs can do, but what they cannot.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tiglionabbit on Saturday October 08, 2016 @03:07AM (#12665996)
I'm curious what Firebase users do in real life. Do people realize this is a problem?

I can imagine making an app like that where all mutations go through my server and reads go through Firebase. How would this interact with the offline persistence feature though? I suppose you could allow the client to do some simple mutations like editing text fields. For the mutations that must go through the server, though, perhaps you could do optimistic local updates?

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tlarkworthy on Sunday October 09, 2016 @12:42PM (#12671954)
The latency compensation through the sdk is optimistic writes. If you shortcut to server, you have to reimplement that so that's not a good solution for many cases. It would confuse the offline persistence too.

Instead, your server should listen to the Firebase, rather than have writes go a separate path (which loses lots of useful features). I think the term now is a client-database-server architecture.

it's not actually a problem for 90% of apps. It's usually in the users best interest not to break their relationships in the db. Think chat. Think info services like Pokemon. Cross table transactions are relative minority in the space of useful apps that's can make money. For those more complex domains, you need technical know how, and Firebase is a useful system component in a larger architecture.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tiglionabbit on Sunday October 09, 2016 @03:22PM (#12672887)
As soon as you want to view something through a different direction you run into this problem. Sure, you can embed chat messages into a conversation, but what if you want to look up every message by a particular user?

I would be surprised to find that 90% of apps don't have this kind of requirement. Pokemon is easy because that data is static.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tlarkworthy on Monday October 10, 2016 @11:23AM (#12677560)
Viewing all posts by a user is solved with classic denormalization. You maintain a separate index of post by user and do two writes per chat msg. It's a case where a user has no incentive not do the second write so it be fine to implement clientside. If however it was business critical, you would have a server side process listen to all chat messages and write the user to message index with the admin account. That index would essentially be eventually consistent.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by williamstein on Friday October 07, 2016 @09:50PM (#12665231)
[edit: this is an irrelevant remark] RethinkDB supports joins: https://www.rethinkdb.com/docs/table-joins/ "Like many traditional database systems, RethinkDB supports JOIN commands to combine data from multiple tables. In RethinkDB joins are automatically distributed—a join command is automatically sent to the appropriate nodes across the cluster, the relevant data is combined, and the final result is presented to the user."
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by Jweb_Guru on Saturday October 08, 2016 @01:38PM (#12667798)
Rethink joins aren't consistent (in fact, no Rethink multikey transaction guarantees any degree of consistency), so even though the original conversation was about Firebase IMO it is just as applicable to Rethink.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by tiglionabbit on Friday October 07, 2016 @09:51PM (#12665234)
I'm talking about Firebase :P.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by dsl on Friday October 07, 2016 @08:24PM (#12664921)
I searched and searched, but I couldn't find a download link on Firebase.com. Help?
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by williamstein on Friday October 07, 2016 @09:09PM (#12665102)
I think Firebase is a closed source program that can only be used via "the cloud", i.e., via a web application. It's not something you can run yourself. In contrast, RethinkDB+Horizon would have soon provided similar functionality to Firebase, but with the option of self hosting.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by bjt on Friday October 07, 2016 @06:29PM (#12664408)
Has Firebase been through Aphyr's Jepsen tests?
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by pritambaral on Saturday October 08, 2016 @03:54PM (#12668434)
Seeing as how Aphyr can't deploy his own instances of Firebase to simulate network partitions, I don't think it has.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by thesmallestcat on Friday October 07, 2016 @05:34PM (#12664087)
Pretty much anybody can appreciate that, right? Not just the "deep thinking people"?
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by karlgrz on Friday October 07, 2016 @08:30PM (#12664935)
Concur. We've been running a small cluster for about a year and we've had the same experience thus far.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by sandGorgon on Saturday October 08, 2016 @05:07AM (#12666230)
isnt that what Realm is trying to do on the mobile side ?
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by jdale27 on Friday October 07, 2016 @05:52PM (#12664194)
"Worse is Better" triumphs again...
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by pan69 on Friday October 07, 2016 @06:21PM (#12664365)
Just going out on a limb here, and I know neither MongoDB or RethinkDB (nor the companies behind them) very well.

But, from what you're saying it almost sounds like that RethinkDB would be a great companion to MongoDB. And if so, that the company behind MongoDB might be a great custodian for RethinkDB.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by davidbanham on Friday October 07, 2016 @06:33PM (#12664435)
That thought fills me with dread. Rethink is well engineered and has a great developer experience. Mongo isn't and doesn't.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by DigitalJack on Friday October 07, 2016 @07:14PM (#12664646)
All these comments are starting to make me feel stupid. I picked mongo for a side project, and it's been really nice so far. I have a hard time imagining how one would use it for a complicated and/or large schema though. I assume that is just my newness to it.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by jpgvm on Friday October 07, 2016 @08:43PM (#12664991)
Don't feel bad for not understanding how to make it work for a big project, it just doesn't. You can try make it work for a big project, but at that point you are just implementing database level code in your application.

Better to just use something like PostgreSQL. Contrary to popular belief PostgreSQL is actually a better document store than MongoDB via it's jsonb type, operators and GIST/GIN indexing.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by __jal on Friday October 07, 2016 @07:52PM (#12664809)
Nope, you have it right. Mongo is a lot of fun to play with out of the box. Once you have nontrivial data models, redundancy and load, not so much.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by pan69 on Friday October 07, 2016 @07:26PM (#12664693)
I'm not entirely to sure if I understand your comment, however, what I mean is;

The company behind MongoDB is a database company so they probably have some sort of affinity and appreciation for a product like RethinkDB, and, they are commercially successful (or at least, it appears to be that way).

Just because, hypothetically, MongoDB could be a custodian for RethinkDB don't mean that all of a sudden RethinkDB becomes MongoDB.

As Paul pointed out, both solutions seem to be two different products covering two different use cases and audiences and from what I can imagine, they might actually complement each other.

Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by meira on Friday October 07, 2016 @06:41PM (#12664494)
MongoDB was much more breakthrough than this little thing called rethinkdb. Hell, people even create the term NoSQL because of it. Maybe the failure of this project came from this dellusional thinking I keep reading in all posts about their failure.
Re:RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Op... by jpgvm on Friday October 07, 2016 @08:45PM (#12664997)
MongoDB coined the term maybe, along with webscale and a whole bunch of meme.

But "NoSQL" atleast as it'd defined has been around forever, it predates SQL because everything before then was trying to work out how to store bits on disk as efficiently as possible before relational theory really found it's way into storage systems and revolutionized things by focussing on modelling.

Even putting aside that fact it was a terrible database then, it's a slightly less terrible database now and everything it can do is done better by other databases.