Sheldon_fun
I built Foyer: a Rust hybrid cache that slashes S3 latency
Powering Crypto Workloads with Sub-100ms Latency
Forget Vector Databases: RAG with Just SQL and LLM
Mitigating Backpressure from High Join Amplification with Unaligned Joins
Re:Data Engineering 2025: Unified Batch‑Streaming, Iceberg Rise and Da...
on Wednesday August 20, 2025 @11:17PM (#44968718)
I’m curious: Have you started using data contracts in your pipelines? Is the unified batch/stream model worth the added complexity?
Data Engineering 2025: Unified Batch‑Streaming, Iceberg Rise and Data Contracts
Re:The Equality Delete Problem in Apache Iceberg
on Thursday August 14, 2025 @11:35AM (#44901700)
From my private conversations with several Iceberg PMC members, it’s clear that full equality delete support across major query engines will be slow — not due to lack of will, but due to complexity.
The Equality Delete Problem in Apache Iceberg
Re:Why Postgres CDC to Iceberg isn't a solved problem: lessons from pr...
on Tuesday July 29, 2025 @12:15PM (#44725182)
Although the last time I touched Debezium in 2020, it was too immature to adopt, I thought surely its problems had been solved by now. Apparently not. I really appreciate this in-depth list of real-world problems encountered by clients trying to pipe CDC-captured changes.
Why Postgres CDC to Iceberg isn't a solved problem: lessons from production
Creating a Streaming Iceberg Table in 3 Steps with RisingWave
RisingWave: An Open‑Source Stream‑Processing and Management Platform
Rust hybrid memory and disk cache to cut S3 latency and costs
RisingWave Adds Hosted Iceberg Catalog– No External Setup Needed
Foyer aims to be an efficient and user-friendly hybrid cache lib in Rust
Nimtable: Open-source web UI to browse and manage Apache Iceberg tables
An On-Premise Observability and Management UI for RisingWave
We Built an In-Memory-Class Architecture on Top of S3 – and Made It Work
Re:We Built an In-Memory-Class Architecture on Top of S3 – and Made It...
on Wednesday May 14, 2025 @12:53PM (#43986715)
No fluff, no hand-waving—just every mistake, lesson, and trick we learned along the way.
Detect Spoofing in Real Time Using Just SQL and Open-Source Tools
Re:
on Friday April 11, 2025 @12:04AM (#43650417)
Understanding Twitter's Data Infrastructure Challenges and Open-Source Solutions
The Rust Closure Pitfall That Silently Corrupted Our Metrics: A Debugging Saga
Re:The Rust Closure Pitfall That Silently Corrupted Our Metrics: A Deb...
on Tuesday April 08, 2025 @11:19PM (#43628589)
Key takeaways:
Rust 2021's closure minimal capture breaks RAII patterns when only Copy-type fields are used in structs with Drop impl.
Even with impl Drop, closures may capture Copy fields instead of the whole struct — a surprising edge case.
Fix requires explicit ownership transfer via let stats = self.stats to override closure's partial capture.
Show HN: Await-Tree – Visualize Async Rust Task Execution in Real-Time
Ending Flaky Tests in Distributed Systems via Deterministic Simulation
BigQuery charged us $10k in 22 seconds for a query with LIMIT 1M – here's why
Re:We Ditched Etcd for PostgreSQL in Our Cloud-Native Streaming Database
on Tuesday March 25, 2025 @11:36AM (#43472609)
etcd is primarily designed for bare-metal deployments, and its performance often suffered in cloud environments due to the relatively slower disk performance compared to on-premise setups.