News for hackers. Stuff that matters.
Re:RethinkDB raises an $8M Series A by lincolnq on Monday December 16, 2013 @12:17PM (#6916166)
Rethinkdb is a really well-designed system. I've been using it (not in production currently) as a better-designed MongoDB with a proper query language. I would recommend checking it out for new projects where a document datastore is appropriate, and to migrate away from a troublesome Mongo.
Re:RethinkDB raises an $8M Series A by octix on Monday December 16, 2013 @12:21PM (#6916191)
I've been using mongo for a few years now, can you please elaborate more on better designed mongo and troublesome mongo?
Re:RethinkDB raises an $8M Series A by boomzilla on Monday December 16, 2013 @01:26PM (#6916704)
Three words: JOIN, JOIN, JOIN. For any non-trivial domain, you'll need to join different entity types. For example, a collection of users is fine for user names, (hashed) passwords, last logins, etc. A collection of document is fine for text, fonts, URL. Now what happens if you have multiple users collaborating on multiple documents? Are you going to denormalize the shared documents to user objects, or are you only keeping document IDs in them? If you choose the former, you'll run into inconsistency very quickly. If you choose the latter, you'll need join support, or you have to write the code to do the poor man's join which could be very inefficient.

RethinkDB supports join out of the box.

Re:RethinkDB raises an $8M Series A by octix on Monday December 16, 2013 @02:21PM (#6917141)
So, if I have 30 nodes and my data is sharded between them, how is that better than having app/client side join, where I have more control what to fetch and what not? I could potentially cache results.

I mean, you realize if data is distributed at large scale, it may take a while till it gets from all nodes the data and joins it...

Re:RethinkDB raises an $8M Series A by boomzilla on Monday December 16, 2013 @02:54PM (#6917365)
If you let the DB do the joins, it could handle more efficiently. For example, it could distribute the joins to those 30 partitions of the main table, and then merge the results, so the heavy computation is distributed, and less bits to move around the network.

Now in the cases where if you can optimize the joins, you still have the option of doing it in your code in RethinkDB/CouchDB. I've done that too, and it's usually when I know for sure that I can prune a big collection to a very small subset more efficiently than using an index.

I would still argue that client app is not the right level of abstraction for data join though, unless it is a big performance gain for very little extra complication.

Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @03:17PM (#6917514)
Slava @ rethink here. I'd be curious to see a use case where doing the join on the client is more efficient than doing it on the server. I can't think of a single one off the top of my head.
Re:RethinkDB raises an $8M Series A by SamReidHughes on Monday December 16, 2013 @09:34PM (#6919257)
If the join is "compute all f(s, t) for s in S and t in T" then you'll save bandwidth (having O(|S| + |T|) bandwidth over the network instead of O(|S||T|)) by doing it on the client. Of course you could just run `rethinkdb proxy` if you want to save ethernet bandwidth and run the query on RethinkDB while connecting to the local cluster node.
Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @10:24PM (#6919413)
Ah, I see, that makes sense. Usually pulling both tables in full (or even a single table in full) to the client is not an option (and nobody does cross products in real-time systems). So people end up pulling a subset of table A, and then for each document in the subset issue a separate get to the db for table B (which is obviously worse than having the db do it).
Re:RethinkDB raises an $8M Series A by fourstar on Monday December 16, 2013 @02:24PM (#6917173)
I actually have been running into this a lot with my documents in my project. Thinking about making the switch to Rethink. The only thing I'm bummed about is having to migrate all my Mongo code over. Anyone here end up doing something similar?
Re:RethinkDB raises an $8M Series A by simonrobb on Tuesday December 17, 2013 @01:06AM (#6919813)
This was brought up in a big way in the recent firestorm around Mongo, but the lack of joins precludes it from any use as a relational database (which is fine, since it's not intended as one). However, I've never been able to use Mongo as a large-scale store as a result.

It seems like the addition of joins lends some relational capabilities which in my eyes is very impressive. I'll be watching the advance of this one with interest.

Re:RethinkDB raises an $8M Series A by nawitus on Monday December 16, 2013 @12:49PM (#6916409)
How does Rethinkdb handle replication, multi-master and MVCC? Thath's the main reason to use CouchDB over MongoDB.
Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @12:56PM (#6916457)
Slava @ rethink here.

RethinkDB supports MVCC out of the box -- you don't have to do anything as a user to get it. (It basically means you can do writes and long reads concurrently without any locking issues).

Sharding and replication in Rethink is similar to Mongo's architecturally, but is much easier in practice. You can shard and replica in Rethink in one click -- check out the 1m highlights video (http://rethinkdb.com/videos/what-is-rethinkdb/) to see how it works. If you have a little more time, take a look at the 10m screencast for more details -- http://rethinkdb.com/screencast/.

Re:RethinkDB raises an $8M Series A by embwbam on Monday December 16, 2013 @01:44PM (#6916870)
I was talking to a guy from Mongo at a conference and he said that Rethinkdb's sharding is as bad as Mongo's because it can't be done in realtime. Is this true? Do you think he meant that it would just be slow to move to sharding for a big database?
Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @02:02PM (#6917004)
In practice today, Rethink's sharding is very similar to Mongo's sharding. This won't remain the case for long -- you'll soon be able to add and remove shards live if that's a requirement in your application. The architecture's already in place, but there are some loose ends we have to tie up to expose this functionality to users.
Re:RethinkDB raises an $8M Series A by ddorian43 on Monday December 16, 2013 @03:31PM (#6917606)
sharding IS NOT similar to mongo, although it's easier, non-primary key sharding + range sharding(which is going away) is missing

and i think those are very important in high-performance sharding (think-per user_id sharding etc)

Re:RethinkDB raises an $8M Series A by willvarfar on Monday December 16, 2013 @01:24PM (#6916682)
Is that ACID transactions with multiple keys?
Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @01:37PM (#6916817)
No, there is no ACID for multiple keys/documents, only on a single key/document.
Re:RethinkDB raises an $8M Series A by pspeter3 on Monday December 16, 2013 @12:54PM (#6916451)
How do CouchDB and RethinkDB compare?
Re:RethinkDB raises an $8M Series A by rkalla on Monday December 16, 2013 @06:10PM (#6918499)
In addition to what other's posted, CouchDB is one of the few NoSQL solutions that will give you master-master replication.

Mongo and Rethink are both single-master/multi-slave solutions.

One is not better than the other, just depends what you need.

And as someone else mentioned, if you want complexity out of your CouchDB queries, you must write map-reduce functions to provide those "views" for you to query. Rethink you can treat similar to a SQL data store and just execute queries against it.

Re:RethinkDB raises an $8M Series A by JulianMorrison on Monday December 16, 2013 @05:52PM (#6918412)
Couch has a much shorter list of operations. Really only get, put, delete, and whatever you can cook up yourself with mapreduce. Rethink has ad-hoc live queries and joins. I don't think Rethink has a tailable update log in the way Couch does.
Re:RethinkDB raises an $8M Series A by clark-kent on Monday December 16, 2013 @10:46PM (#6919499)
CouchDB has a HTTP REST API, and speaks JSON natively. You can write your own CouchDB client to meet your needs. You can also access your data from any server using http.
Re:RethinkDB raises an $8M Series A by pbogdan on Monday December 16, 2013 @02:30PM (#6917206)
I've been struggling with this one a lot - could someone explain where a document store is a good solution?
Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @02:36PM (#6917242)
Imagine an RDBMS table where many columns are nullable and many rows contain nulls in practice. Dealing with these is really painful in traditional database systems (you don't know how painful until you try a document store).

A document store flips the default. It makes dealing with data that has lots of nullable columns much, much easier. (It also makes dealing with hierarchical data a breeze)

There are lots of details, but this is the gist of it.

Re:RethinkDB raises an $8M Series A by MichaelGG on Monday December 16, 2013 @05:08PM (#6918183)
How is it better than an XML field in SQL Server, which allows indexing, schemas (if you want), and full querying inside the document? I think Postgres also has similar functionality with JSON, now, too.

Certainly a lot of applications would benefit from having a full RDBMS they can opt-in to document-style data when they feel like it?

Built-in horizontal scaling is one selling point for non-RDBMS stores, but large systems seem to just shard on top of RDBMSes anyways, right?

Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @05:17PM (#6918233)
> How is it better than an XML field in SQL Server

It changes the default, which results in a drastically different programming experience. The difference is difficult to describe in the same way a dynamically typed programming language is difficult to describe to someone who's never tried one.

I'd encourage you to try a document store (Mongo, Rethink, whatever) for a throw-away project. A ten minute tutorial walkthrough is worth a thousand HN comments when it comes to stuff like this :)

Here's the Rethink tutorial: http://rethinkdb.com/docs/guide/python/. Just play with it and see if you like it!

Re:RethinkDB raises an $8M Series A by MichaelGG on Monday December 16, 2013 @07:03PM (#6918801)
OK, I will try it out for something and see how it feels different.

Related: The comparison to a dynamically typed language makes me suspicious. I spent a bit of time trying to find any examples of dynamic code that actually provided any benefit. Even read "Metaprogramming Ruby" and was dismayed to see examples of reading a CSV - big deal if I save a few quotation marks. The others were just places where the static type system wasn't good enough (duck typing), or dynamic code was a pain to get going (poor reflection/codegen APIs).

Re:RethinkDB raises an $8M Series A by nostrademons on Monday December 16, 2013 @08:52PM (#6919146)
I'll try to add some additional color here. My background is that I work for Google (where the vast majority of our data is stored in what amounts to document databases, generally protobufs in a key-value store), I prototype in Python and Javascript, and I write production code in Java and C++.

Both document databases and dynamic typing are at their best when you don't understand your problem domain. They let you express what you do know about your problem domain concisely, and then fill in the blanks later on. So in a document database, when you find that you want to record a new bit of data - just add it as a field to newly-created XML/JSON documents, and only display it in the UI if it's present. Or pick a default value if you need to perform computations with it. Don't bother with data migrations, don't bother with schemas, don't bother trying to backfill previous data. Try out your idea and see if it works first, because chances are, it doesn't.

If you always work on projects where the requirements are handed to you, specs are complete, and the problem domain is understood, this will seem terribly irresponsible to you. And it is - if you understand your problem domain, you should capture as much of that knowledge in the software system you build to understand it.

But if you are working in startups, or in consumer web, where you absolutely have to be on the leading edge or die and the only opportunities that haven't been picked over yet are the ones that nobody understands - being able to try things out without having to flesh out all your assumptions is crucial. You will run circles around the people who spend time defining their data model and speccing out their objects. And then when consumer tastes change - which happens quite regularly - you can adapt to them immediately instead of throwing out all the work you did under the old assumptions.

The other bit of context I'll toss in is to get in the mindset of solving a problem that you don't know how to solve and assume that your first 10 solutions won't work. For example, if you're reading a CSV - everybody knows how to do that, dynamic typing doesn't really help there. If you're cloning Stack Overflow, you can probably figure out what your database schema should be. But what if you're trying to figure out a new way for people to socialize over mobile phones? Where do you start there? That's the use case for dynamic languages and document DBs. The problems where technology is a tool for understanding & manipulating vaguely-defined social behaviors.

Re:RethinkDB raises an $8M Series A by MichaelGG on Tuesday December 17, 2013 @12:59AM (#6919804)
Thank you for the explanation. I can understand the logic.

In F#, adding a field requires "field : type". If I don't care about type checking, I can just add "Props : dict<string, object>" and go to town. Or I can opt-in to the dynamic features and just do "foo?bar <- baz". When I change types around, things either just work due to type inference, or the compiler helpfully points out every place that'd be a runtime error. I've never felt this slows me down. I feel the type checking and autocomplete is worth the tiny amount that specifying a record takes. (I've spent days finding minor issues in JavaScript, stuff that'd instantly be caught by a type checker.)

Databases make it a more cumbersome, and it takes more than one line to start using a new field. I totally sympathize with the flexibility issue there. Even with a document type, most syntax I've seen doesn't have truly first-class querying support (not as easy a column, anyways). And it feels ugly to have some fields defined in schema, and some in a document. But that seems like a minor tooling issue -- there's no fundamental reason SQL can't let me do "WHERE x.SomeDoc.SomeField.OtherField > 5" (perhaps some minor scope resolution issues to ensure I'm not referring to some other multi-part name).

Re:RethinkDB raises an $8M Series A by baruch on Monday December 16, 2013 @02:38PM (#6917258)
I'm using it to collect stats from various systems, events and alerts too. It makes it easy to collect new data points and add fields to new events and query things in a consistent way across the entire data set.

When the schema is not rigid and likely to change on a daily basis I prefer a document database over an SQL one. There are probably other use cases as well, this is just my favourite.

Re:RethinkDB raises an $8M Series A by codelust on Monday December 16, 2013 @04:10PM (#6917827)
One example is a CMS. An article these days is a title, body and 5-10 comments, and whatever other metadata that you would want to present on a page. Don't update the corresponding NoSQL record till there are writes in the RDMBS. Serve from the NoSQL system, with at least another caching layer in front. You get the best of both worlds and just one query instead of many.
Re:RethinkDB raises an $8M Series A by MichaelGG on Monday December 16, 2013 @07:08PM (#6918824)
Why not just use a materialized view inside your SQL system? No need to cache invalidation; let the system handle it for you. Same benefits, truly a single query, and only one system.

If you're going to concatenate relational data into a document, I'm not sure why a simple KV table doesn't fit the job.

Re:RethinkDB raises an $8M Series A by codelust on Tuesday December 17, 2013 @12:58AM (#6919800)
Thanks, had to read up on what exactly is a materialized view. Not something I was familiar with.
Re:RethinkDB raises an $8M Series A by jnbiche on Monday December 16, 2013 @03:24PM (#6917567)
I've been looking at Rethinkdb for an upcoming project that has a decent amount of relational data. Rethinkdb seems a lot different from Mongodb in that it's partially designed to accomodate relational queries. Has that been your experience?

Also, does Rethinkdb have any kind of transactional capabilities?

Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @04:30PM (#6917948)
> Does Rethinkdb have any kind of transactional capabilities

You have full ACID on a single document, but not across multiple documents. In this way Rethink is similar to other NoSQL systems (except you can do almost any operation imaginable atomically on a single document in RethinkDB).

Re:RethinkDB raises an $8M Series A by amenod on Monday December 16, 2013 @04:05PM (#6917800)
Nice, can't wait to try it out! Though in my case (single node installations) hot backup missing feature is a deal breaker, so I'll have to wait till RethinkDB supports it. The current workaround (replication, stop the slave, backup, start the slave) is just not good enough for me...
Re:RethinkDB raises an $8M Series A by coffeemug on Monday December 16, 2013 @04:24PM (#6917904)
Hot backup has been available since 1.7. You can learn more about how it works here: http://rethinkdb.com/docs/backup/

Enjoy!