Yes you can try to do bunch of hacks like that, but even just VM failure rate in cloud is insane compared to any modern hardware (looking at you, gcp, but aws too and lower tier providers are even worse). I see multiple VMs drop ded every week with no way to save those logs unless you use PDs ($$$)

Exports your logs to where? I would call that place centralized logging.

I’m assuming neighboring nodes. Now you’re basically designing replicated (best effort), distributed index - a mini version of google search. This is a clever design but number of companies that can diy it with sufficient quality I can probably count on one hand

For the price of even a modest splunk enterprise setup you can hire half a dozen top notch backend/distributed systems engineers. That’s the opportunity cost of a setup like the one in the article.

Right if talking top notch that’s 3-4M/y alone. Admittedly I’m completely unaware what the splunk’s ballpark rate is (i heard it’s quite high) but still I doubt that very many companies are at that scale. Also in the current market convincing execs to save on something that doesn’t translate directly to cogs can be a challenging proposition =)

I’ve heard this before and it sounds like Splunk’s margins are just way too high. Why doesn’t someone sell a logging platform that’s cheaper than writing your own at scale?