News for hackers. Stuff that matters.
Re:Simple Systems Have Less Downtime (2020) by eatonphil on Wednesday August 04, 2021 @09:54AM (#28061023)
OK well I'm telling you based on my experience and it wasn't at FB or Google. :) Do most people have these problems, no you're right. Should everyone ignore them and be unaware of options when they do have the problem? Probably not?
Re:Simple Systems Have Less Downtime (2020) by LeonB on Wednesday August 04, 2021 @11:07AM (#28061849)
I had those problems several times within the last few years - having to aggregate and detect various signals from billions of data points - and keep the system flexible enough that different business people could “try out” various ideas and play with the results.

All sorts of different patterns and architectures have to be brought together to make sense of it.

But the technique described above still has its place within such a system. For example - you boil billions of data points down to a handful (several thousand) facts, such as events that occurred or anomalies. Then those several thousand facts can be analysed standalone (as described above) without going back to the source. Etc.