News for hackers. Stuff that matters.
Re:MongoDB's lead developer: Foursquare outage post mortem by ww520 on Thursday October 07, 2010 @09:34PM (#1770243)
This is similar to one of the problem I used to deal with in tracking huge volume of billing events. Since the data (checkins) are immutable, it's fairly easy to build summary running totals daily. The total count then becomes the running-total plus the current day's count. The summary job can be run offline in the background. Just make sure to mark the current day's events as processed and update the running total at the same transaction.

In this way, the amount of current day's events are pretty small and you don't need to keep all historic events in memory.

There are no comments on this comment yet.