Re:RethinkDB versus PostgreSQL: my personal experience
by osullivj on Friday February 10, 2017 @05:34AM
(#13614472)
Raw insert performance is the main driver for tick databases persisting high frequency market data for HFT systems. The TimesTen DB, later bought by Oracle, was built for this scenario. Arthur Whitney's KDB too.
Re:RethinkDB versus PostgreSQL: my personal experience
by StreamBright on Friday February 10, 2017 @01:27PM
(#13617639)
Not to mention the sub second latecy for querying 1B rows.
Re:RethinkDB versus PostgreSQL: my personal experience
by pmelendez on Friday February 10, 2017 @05:45AM
(#13614511)
Not the single dimension but an important one. Any real time application processing events data (IoT, heartbeats, prices logs) would have a similar requirement. That's one reason why people uses queues, to try to workaround this. And as for the reads, it will drilldown to your query needs. I have done systems where the read requirement was a retrieval from a key and the writes request came in thousands per second.
Re:RethinkDB versus PostgreSQL: my personal experience
by StreamBright on Friday February 10, 2017 @01:33PM
(#13617701)
Absolutely, I have run data ingestion pipelines before for event data processing, perfectly suitable for the use cases you mentioned too. I just dont see mongodb as a good fit here.
Hint: you probably can't, the reason why you store data is that you can read it (query it) later. MongoDB could be 10000x faster with inserts, still not a considerable solution for us because it fails with many of the other aspect of providing a great data access layer.