News for hackers. Stuff that matters.

A comment on Show HN: Open-source serverless security lake powered by Rust + Apache Iceberg. Parent comment

Re:Show HN: Open-source serverless security lake powered by ... by shaeqahmed on Saturday August 06, 2022 @01:13PM (#32369815)
Using and maintaining Matano is a fraction of the cost compared to popular non-serverless alternatives like ELK or Spunk. Matano is specifically designed for petabyte-scale security analytics use-cases that don't fit in a traditional SIEM.

The serverless data ingestion pipeline means you don't need to over-provision for ingestion (Logstash and Splunk Forwarders are notorious for related costs / ops in high scale use-cases) in the write path. For reads, since Matano queries Iceberg tables backed by highly-compressed parquet files on object storage you won't pay anything close to what you would for a database or search engine based SIEM.

[ Parent | on HN ]
Re:Show HN: Open-source serverless security lake powered by ... by mdaniel on Saturday August 06, 2022 @04:55PM (#32371518)
> For reads, since Matano queries Iceberg tables backed by highly-compressed parquet files on object storage you won't pay anything close to what you would for a database or search engine based SIEM

Where do you show an example of querying anything? There's an empty "detector" in the examples directory, which I guess gets called once per row of this 20MiB/s alleged elsewhere?

Anyway, I find comparing this to Splunk to be a bit premature

[ Parent | on HN ]
Re:Show HN: Open-source serverless security lake powered by ... by FridgeSeal on Sunday August 07, 2022 @12:46AM (#32373974)
Tools like Spark, Trino, etc can be pointed at parquet/iceberg/etc files in S3, and they'll let you issue SQL queries against the files directly. Means it integrates out of the box with whatever data tooling is being used in your org already.
[ Parent | on HN ]