News for hackers. Stuff that matters.
Re:Beringei: A high-performance time series storage engine by cocoflunchy on Saturday February 04, 2017 @12:30PM (#13567875)
Yes, we only have one DB for everything and it is bottle-necking on writes. Queuing the writes is probably the easiest solution since we already have RabbitMQ setup... thanks for your answer.
Re:Beringei: A high-performance time series storage engine by saosebastiao on Saturday February 04, 2017 @01:27PM (#13568185)
There are a lot of good suggestions here, but if this is the route you're going down and you are still using the same database, do everything you can to batch your inserts. In your queue consumer, aggregate as much as you can tolerate in memory, and then insert them all and commit the inserts in a single transaction. This eliminates a huge amount of MVCC overhead.