mineev
Re:Can you build observability ingestion on S3 alone – no Kafka, no di...
on Tuesday July 07, 2026 @05:06PM (#48823758)
You are right, the time of a single write/read is Achilles' heel. But if we are talking about parallel processing it can keeps several hundreds milliseconds delay and provides enormous throughput.
Re:Can you build observability ingestion on S3 alone – no Kafka, no di...
on Tuesday July 07, 2026 @04:11PM (#48823035)
Actually, my question not about possibility, but is there any value in it? May be any message queue or k8s cluster with StatefulSets is easier to operate? What do you think?
Can you build observability ingestion on S3 alone – no Kafka, no disks?
Re:Show HN: IceGate – Observability data lake engine
on Monday April 13, 2026 @11:17AM (#47753265)
Why it matters:
→ 3x faster than nearest competitors. Rust + Apache DataFusion + Arrow do the heavy lifting.
→ Exactly-once delivery with WAL. No data loss, no duplicates - guaranteed.
→ Your data stays in open formats you control. No vendor lock-in. Ever.
→ Query logs through Loki-compatible API - works with your existing Grafana setup.
→ Ingest via standard OpenTelemetry (gRPC/HTTP). No proprietary agents.
→ Your data is in Iceberg tables - query it with Trino, Spark, Flink, DuckDB, or any tool in the Iceberg ecosystem.
→ Your observability data becomes a first-class asset of your data lake alongside business analytics.
→ Scale ingest, query, and storage independently. Full compute-storage separation provides stateless deployment.
Show HN: IceGate – Observability data lake engine
https://github.com/icegatetech/icegate
Today we're (@sprosvirnin and I) launching the first open-source release of IceGate - a Rust-native engine that stores logs (metrics and traces coming soon) in Apache Iceberg tables on your own object storage.Observability costs are out of control. Teams pay per GB ingested, get locked into proprietary formats, and can't use their own data outside the vendor's UI. We built the first...