Eg. Dump all of the data to S3 and search it in parallel via AWS Athena.
If you want fast fuzzy search, then elasticsearch is built for this.
If the data is small enough that you want B-tree indexes (with the accompanying slower writes and lower query latencies), then any SQL database with a JSON column will work.
Too proprietary. Can't be run locally, on robots, etc. Can't be transferred to other cloud providers. Also S3 is horribly slow to write or delete thousands of records at once.
Every major cloud provider has a blob store. Most are S3-interface compatible, or can be fronted by something which is, like Minio.
> Can't be run locally, on robots, etc.
If the contents of your S3 keys are just line-delineated JSON files, you can easily download those files and run scripts or process them locally.
> Can't be transferred to other cloud providers.
Again, not true — a tool like rclone handles this case in a single command. Depends on the amount of data, but if anything it's easier to move a bunch of flat files between providers than copying a database backup around. You have to pay for egress bandwidth, though, of course.
> Also S3 is horribly slow to write or delete thousands of records at once.
If you're keeping one JSON record per S3 key, that is a blatant misuse of the tool and performance will be terrible. On the other hand, if you batch records into files of appropriate sizes, it's very cost-effective to store and query via Athena, Spark on EMR, etc.
If you need individual record-level access as identifiable by a key, then you'd probably be better off with Redis or Memcached. (though, those will not be as good for bulk offline processing) It's all about your access patterns.
Too much work. If you're a startup that's too much stuff to maintain. Much easier to get a hosted MongoDB and launch tomorrow.
And when you only have 2 months rent in the bank and investors want demo after demo in order to give you cash, tools like MongoDB do make a difference. And yes, I've been in that situation before.
I'll add that S3 is only slow to delete things sequentially. You can delete or write an effectively infinite amount of items in parallel.
That's what interface contracts and API versioning is for - and new fields shouldn't be any problem at all unless you're parsing the JSON by hand.
When you have multiple types of objects with differently populated fields that you want to store in a single collection and query across them in currently unknown ways, and where you need flexibility for the unknown.