News for hackers. Stuff that matters.
Re:Scaling with MongoDB (or how Urban Airship abandoned it f... by mathias_10gen on Wednesday June 22, 2011 @08:45PM (#2686120)
FYI - Your _id trick is similar to the ObjectID type mongodb uses by default.

"A BSON ObjectID is a 12-byte value consisting of a 4-byte timestamp (seconds since epoch), a 3-byte machine id, a 2-byte process id, and a 3-byte counter. Note that the timestamp and counter fields must be stored big endian unlike the rest of BSON. This is because they are compared byte-by-byte and we want to ensure a mostly increasing order."

http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs-BSO...

Re:Scaling with MongoDB (or how Urban Airship abandoned it f... by FooBarWidget on Thursday June 23, 2011 @04:02AM (#2687095)
My _id is different from ObjectID. It does begin with a timestamp but one that has 2010 as epoch. It's also followed by a kind of user ID and a piece of random identifier that appears elsewhere in the document.
Re:Scaling with MongoDB (or how Urban Airship abandoned it f... by tszming on Wednesday June 22, 2011 @10:22PM (#2686375)
Interesting, so the _id trick mentioned by FooBarWidget is not the real reason for the speedup?
Re:Scaling with MongoDB (or how Urban Airship abandoned it f... by FooBarWidget on Thursday June 23, 2011 @04:03AM (#2687097)
It is. I was using a totally random string key, not the default ObjectID.
Re:Scaling with MongoDB (or how Urban Airship abandoned it f... by tszming on Thursday June 23, 2011 @09:58AM (#2687849)
So if you stick with the default _id, the claim of "Your indexes must fit in memory" is no longer valid?
Re:Scaling with MongoDB (or how Urban Airship abandoned it f... by FooBarWidget on Thursday June 23, 2011 @10:15AM (#2687918)
That totally depends on your workload. In my case my working set happens to be mostly equal to the most recently inserted data. If you have to regularly access lots of random documents with no locality whatsoever then your working set is very large and should fit in memory.

By sticking with the default _id, with my workload my _id index doesn't have to fit into memory. I can't actually use the default _id for various reasons but that's a whole different discussion.

Re:Scaling with MongoDB (or how Urban Airship abandoned it f... by tszming on Thursday June 23, 2011 @10:41PM (#2690743)
Useful information, thanks!