News for hackers. Stuff that matters.

Migrating a large site from SQL Server to MongoDB - Lessons Learned

SQL and NoSQL are different tools for different jobs. A couple of months ago we migrated a large site from SQL Server to MongoDB, it was a bumpy road and I decided to write this article to share what we learned: http://www.wireclub.com/development/TqnkQwQ8CxUYTVT90/read
( 12 comments | 18 points | on HN )
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by amrith on Sunday November 06, 2011 @05:58PM (#3204184)
I'm shocked and mortified ... Check out http://bit.ly/v1cznd

Why would you conclude that MongoDB was "fast" and SQL Server was slow? Did you run SQL server on the same SSD's and multiple HDD's?

to willvarfar's comments, what was the real fsync cost and didn't you get ACID with SQL Server? Now, if ACID is not really required, and you really really need flexible schemas, then by all means ... I understand that you want distributed-queryable-notepad, not a relational database!

Re:Migrating a large site from SQL Server to MongoDB - Lesso... by rfurlan on Sunday November 06, 2011 @08:24PM (#3204543)
Also, I never said SQL Server is slow - it isn't, anyone that would say otherwise must have an agenda, which is not my case. SQL Server is an incredible database and it has served us well.
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by rfurlan on Sunday November 06, 2011 @08:20PM (#3204531)
MongoDB has the potential to be faster than traditional relational databases simply because it does less, way less. The trade off makes sense for some applications, but not for all, it really depends on your requirements.
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by rfurlan on Sunday November 06, 2011 @03:46PM (#3203776)
I also hosted a Q&A about our experience here on HN: http://news.ycombinator.com/item?id=3141331
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by byoung2 on Sunday November 06, 2011 @03:01PM (#3203642)
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by rfurlan on Sunday November 06, 2011 @03:45PM (#3203772)
Thanks byoung2!
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by willvarfar on Sunday November 06, 2011 @03:48PM (#3203785)
So do you have fsync enabled?
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by rfurlan on Sunday November 06, 2011 @03:52PM (#3203800)
fsync is enabled for the vast majority of calls and it is our default setting. Disabling fsync should be treated as case-by-case optimization IMO
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by latch on Monday November 07, 2011 @06:28AM (#3205835)
I assume when people talk about fsync, they actually mean journaling?

I'm not sure if there's ever a good reason to use {fsync: true} vs {j: true}

Re:Migrating a large site from SQL Server to MongoDB - Lesso... by rfurlan on Monday November 07, 2011 @12:10PM (#3207097)
Fsync means "wait for acknowledgement after each request", if Fsync is off, all calls are fire-and-forget, no errors are reported. There are a few Fsync levels though, you can use it to make the call block until the data has been relayed to at least N other nodes.
Re:Migrating a large site from SQL Server to MongoDB - Lesso... by latch on Monday November 07, 2011 @08:23PM (#3209040)
no, fsync means wait for the data file to be synced to disk. {safe:true} is what you are talking about. {j:true} is probably what yo should be using, {w:majority} is also important, and {fsync:true} is the data file option

I think given that one of the parameters is called "fsync" using "fsync" should mean that.

Re:Migrating a large site from SQL Server to MongoDB - Lesso... by rfurlan on Monday November 07, 2011 @08:27PM (#3209049)
Yes latch, you are correct, thank you