News for hackers. Stuff that matters.
Re:Beringei: A high-performance time series storage engine by zo1 on Saturday February 04, 2017 @12:39PM (#13567915)
Don't write it to your main transactional database, then.

Have a separate connection to a DB on another machine that is handling the load. Do that within the transaction/request from the client, or pop it into a message queue.

The queue will "persist" the data, but will also schedule it for when your other box/DB can ingest it. At the end of the day, if you're pushing too-much data, you need to have a mechanism in place to let you scale. To me, a simple solution would be a message queue server with plenty of redundant storage space. From there on, you keep adding more consumers.

Re:Beringei: A high-performance time series storage engine by GordonS on Saturday February 04, 2017 @12:46PM (#13567959)
If the message queue becomes a bottleneck, buffering/batching can also help - instead of sending individual messages to the queue, send them in batches (based on the number, time period, or both).