Re:MongoDB has filed confidentially for IPO
by tehlike on Wednesday August 16, 2017 @02:11AM
(#15025296)
sounds like the wrong tool. what you are producing was probably logs data, which is immutable. There are far more efficient (storage, cpu) write-only stores.
[ Parent
| on HN ]
Re:MongoDB has filed confidentially for IPO
by skrebbel on Wednesday August 16, 2017 @02:45AM
(#15025422)
Such as?
Re:MongoDB has filed confidentially for IPO
by hvidgaard on Wednesday August 16, 2017 @03:00AM
(#15025482)
Concider what a database does. It provides ACID properties and ability to query data. If all you need is writing data, the fastest you can do it write it directly to the disk, without the overhead a database comes with.
Using a loadbalancer in front of a farm of cheap logging machines, and aggreate the data you need for analysis to a suitable machine.
Re:MongoDB has filed confidentially for IPO
by LoSboccacc on Wednesday August 16, 2017 @04:30AM
(#15025847)
they'd still need to query their stuff, I guess, so you'd need to trow in there somewhere something to aggregate logs and get the metrics they're tracking out of it - which can totally be done in streaming, without the need of going trough the logs every time, for most metrics.
Re:MongoDB has filed confidentially for IPO
by hvidgaard on Wednesday August 16, 2017 @04:42AM
(#15025904)
With that amount of data, streaming and only saving aggregated data is the only sane way. With 1.2TB/hour there is a limit to how much historical data that can be saved anyway, and we're talking about 30% utilization of a 10gbps network interface, so it's beyond using single machines for most usecases.
Re:MongoDB has filed confidentially for IPO
by tehlike on Wednesday August 16, 2017 @06:16AM
(#15026317)
Query? Most likely not. At least not in the traditional "lets on the fly create a dashboard" sense.
Re:MongoDB has filed confidentially for IPO
by LoSboccacc on Wednesday August 16, 2017 @06:20AM
(#15026337)
query as in 'how much I bill this guy for it's click' - doesn't have to be sql nor on the fly of course