News for hackers. Stuff that matters.
Re:Logging at Twitter by jcims on Saturday February 19, 2022 @12:29AM (#30394667)
We log >>100TB/day in splunk and get all the discounts. It’s still ridiculously expensive.

Many of the issues presented in the article ring very true. Splunk is pretty amazing for adhoc analysis/threat hunting. However, once you know what you’re looking for the value proposition drops precipitously.

Re:Logging at Twitter by romantomjak on Saturday February 19, 2022 @04:42AM (#30395730)
Can you give a ballpark figure? I can’t even begin to imagine how much that costs.
Re:Logging at Twitter by jcims on Sunday February 20, 2022 @10:10AM (#30406276)
All in (software/hardware/people) ~$1-2k per terabyte of data indexed.
Re:Logging at Twitter by halilduygulu on Monday February 21, 2022 @04:58PM (#30421171)
what kind of use case can log 100tb? Do you actually query this data later?