ciaranlee
The DynamoDB move is certainly a tradeoff, but for us it really makes sense. Moving to it will allow us to:
- scale out this storage layer trivially easily
- shed a ton of operational responsibility
- save lots of money
There are certainly downsides to moving to a proprietary database run by a single proivider, but we’re happy to make that tradeoff.
Also, we’re absolutely not recommending that people move from MongoDB to DynamoDB as some general rule. It just made sense for us at this time.
In our (Intercom) case these tasks are not simple.
We are lucky to have had exponential growth in most metrics (including database throughput) for the entire 7 years of our existence. This is a harsh environment for datastores! A 99.9% SLA makes it even harder.
We are also nowhere near finished building out our vision for the product. We prefer to focus on that instead of operating data stores, which we can pay someone else to do.
Also, you are incorrect about only being able to index two fields in DynamoDB. Though we luckily don’t need anything other than the primary key.
We've lots of interesting things to work on, including:
* Spliting our monolithic rails app in to discrete services
* Redesigning our data storage to handle the next 100x growth
* Improving fault tolerance within the application until there are no single points of failure
* Shrinking the time taken from a push to master until production deployment to less than 10 minutes
* Building tools and processes to help us detect and respond to operational issues quicker
* Increasing visibility in to app performance by working on our graphite or logstash infrastructure
* Designing and rolling out a websocket infrastructure capable of handling hundreds of thousands of concurrent connections