News for hackers. Stuff that matters.
Re:Startup Engineers and Our Mistakes with MongoDB by mrtksn on Wednesday July 19, 2017 @12:56PM (#14805731)
If data needs to be processed, an option would be to use MongoDB for collecting the data in bulk and later decide how you need to structure it for your needs.

When it comes to "read" the data, you read it only from processed database that can be anything.

Since you can do your processing completely independently of your web server, your are not necessarily pushing your computational load from the DB to the web server.

Re:Startup Engineers and Our Mistakes with MongoDB by flavio81 on Wednesday July 19, 2017 @01:02PM (#14805792)
> If data needs to be processed, an option would be to use MongoDB for collecting the data in bulk and later decide how you need to structure it for your needs.

OR, i can open a file stream, serialize my data to JSON, entity by entity, and dump all to a file.

The good old file.

Re:Startup Engineers and Our Mistakes with MongoDB by threeseed on Wednesday July 19, 2017 @01:37PM (#14806142)
Because we want to do partial updates, searches, indexing etc.

Your position can be applied to all databases. Why not abandon them all and just use CSV ?

Re:Startup Engineers and Our Mistakes with MongoDB by bdcravens on Wednesday July 19, 2017 @02:24PM (#14806600)
You could always do that, dump the data to s3, and use Athena :-)

https://aws.amazon.com/athena/

Re:Startup Engineers and Our Mistakes with MongoDB by pritambaral on Wednesday July 19, 2017 @04:03PM (#14807432)
And then be locked-in to Amazon. From ubiquitous files to ... a single service provider.
Re:Startup Engineers and Our Mistakes with MongoDB by mrtksn on Wednesday July 19, 2017 @01:05PM (#14805831)
replied to the same question here: https://news.ycombinator.com/item?id=14805812
Re:Startup Engineers and Our Mistakes with MongoDB by aidos on Wednesday July 19, 2017 @01:10PM (#14805876)
Then again, if you're going to push into Postgres you might as well store the temp unstructured records there too
Re:Startup Engineers and Our Mistakes with MongoDB by mrtksn on Wednesday July 19, 2017 @01:23PM (#14805989)
Well, that's also an option :) But I believe this is a relatively new option.

Personally, I never got into the hype with the NoSQL and for me the only use is as a convenient API for storing JSON files during prototyping so that I can later decide how my data should be structured.

Definitely tech debt for the future but fast iterated design that needs the database stuff fixed has more chance for success than the rigid design with the perfect data architecture.

Re:Startup Engineers and Our Mistakes with MongoDB by emodendroket on Wednesday July 19, 2017 @12:58PM (#14805755)
It that point it's harder for me to understand why I'm not just writing out JSON files or something.
Re:Startup Engineers and Our Mistakes with MongoDB by mrtksn on Wednesday July 19, 2017 @01:03PM (#14805812)
You definitely can but MongoDB provides convenience over storing and managing JSON files on you filesystem by your own efforts.
Re:Startup Engineers and Our Mistakes with MongoDB by lowbloodsugar on Wednesday July 19, 2017 @01:42PM (#14806209)
No, I really can't think of any situation where installing, managing, updating, maintaining, and crying over a mongodb cluster would be easier for a flat-file datastore than say, a file system, or S3. In fact if one is on AWS, then nothing beats just dumping them in S3 and processing them as necessary. If I'm a startup and I really wanted some querying then PostgreSQL looks great, with Amazon handling the maint and backups.
Re:Startup Engineers and Our Mistakes with MongoDB by bdcravens on Wednesday July 19, 2017 @02:26PM (#14806621)
You can even query flatfiles in s3 using SQL via AWS Athena.
Re:Startup Engineers and Our Mistakes with MongoDB by emodendroket on Wednesday July 19, 2017 @01:24PM (#14805994)
Sure, but it is also another service to keep running. Depends on what scale you're operating at.
Re:Startup Engineers and Our Mistakes with MongoDB by mrtksn on Wednesday July 19, 2017 @01:36PM (#14806135)
Any RDBMS is just a toolset to manage files on some filesystem. If you are dealing with some config files you can spare yourself the trouble and just load them from the filesystem.

NoSQL or classic RDBMS, these tools give you ways to manage your data on the filesystem(or in the memory?). You can spend time to write a code that processes files in a folder or you can use the software that does it for you. You can write a code that relates two files in a folder or you can use a software that already can do it and you can just ask that software for the outcome.

Re:Startup Engineers and Our Mistakes with MongoDB by emodendroket on Wednesday July 19, 2017 @02:36PM (#14806702)
I don't really have to write that code though; what language doesn't already have file-manipulation libraries?
Re:Startup Engineers and Our Mistakes with MongoDB by mrtksn on Wednesday July 19, 2017 @02:45PM (#14806780)
Well I guess some people wonder why people are using ready software when you can achieve the exact same thing with a text editor and a compiler.
Re:Startup Engineers and Our Mistakes with MongoDB by emodendroket on Wednesday July 19, 2017 @03:02PM (#14806913)
That's not the same thing at all; you're suggesting an approach that probably takes more code and definitely takes more setup.
Re:Startup Engineers and Our Mistakes with MongoDB by mrtksn on Wednesday July 19, 2017 @04:01PM (#14807412)
You are underestimating the complexities of file management
Re:Startup Engineers and Our Mistakes with MongoDB by emodendroket on Wednesday July 19, 2017 @07:46PM (#14808975)
Again, depends on scope and scale. If you have a handful of items to process it's not a big deal.