smithclay

Surprised that https://github.com/ucbepic/DataAgentBench hasn't been mentioned on this: benchmarking data agents that need to query from multiple sources is a focus of some research at UC Berkeley -- leaderboard is worth a look to see how the various closed and open-source implementations build ontologies ... would like to see how Nvidia's approach stacks up.

It's at the end of this Netflix post: but the end game is to connect whatever your observability ontology is with whatever system the business people and data engineering teams use to be able to answer previously hard questions like "how much money did this operational or engineering change make or lose?"

In theory, at least.

Observability vendors have been beating the drum for 10+ years that there's so much untapped business value in operational data. Seems like we might be getting close to seeing how real that is.

Don't think it's widely known or understood, but Ducklake doesn't require duckdb: it's just a (good) data lake spec that works in duckdb.

There's a cool alternate rust/datafusion ecosystem initiative going on at https://github.com/datafusion-contrib/datafusion-ducklake, and think the Quack protocol opens up a lot of cool possibilities too.

If you need an idea for what do do with ducklake: recommend throwing all of your agent traces in it.

More benchmarks comparing effectiveness of various AI SREs is welcome and overdue: especially vendor-vs-vendor comparisons. One area that I think is going to be really interesting and important is the best way to emulate complex IT environments for evals.

Some related work I recommend checking out: - https://arxiv.org/abs/2609.33023 (new last week!) - https://github.com/SREGym/SREGym - https://github.com/hyperdxio/hyperdx/tree/main/packages/hdx-... (Clickstack's version) - https://github.com/grafana/o11y-bench (Grafana's version)

This is a lot to like here as someone who pretty much exclusively uses the Cloudflare API via agents. Hoping (since it's built on top of Forge), some kind of native terraform support is also in the works.

Great when you're a solo dev deploying a worker, but would be incredible for production deployments if this could also just natively output terraform code.

Feel like AWS got a lot of the concepts right with S3 Tables* ("serverless" Iceberg catalog on top of S3) ... would love, love to see a version of S3 Tables that lets you choose between Ducklake or Iceberg as the catalog.

* Cloudflare also seems to have gotten the same idea right with R2 Data Catalog

As someone who lives in central San Francisco: Costco is amazing with a cargo bike (or Waymo off-peak).

Haven't had Amazon Prime for 2+ years and don't miss it, but would definitely miss my Costco membership.

If you're reading this and curious: consider writing a duckdb community extension* or contributing to an existing one*

duckdb is becoming a kind of data superglue between a lot of data ecosystems (GIS, observability, analytics, lakehouses, object storage, etc) that don't talk to each other typically, and it's worth checking out in 2026.

* https://github.com/duckdb/extension-template * https://duckdb.org/community_extensions/

Not yet.

That said… think duckdb/ducklake/quack could potentially be a future replacement for Mimir or Clickstack with way less operational complexity.

Been working on open-source projects involving storing and querying observability data (metrics, logs, traces) in parquet[0] and have been frustrated with the usability of Apache Iceberg … despite strongly agreeing and wanting to use an open storage format and catalog.

This makes Ducklake much more interesting for my use case, excited where this is going.

[0] https://github.com/smithclay/duckdb-otlp

sprites.dev / fly.io has publicly said they are using a variant of JuiceFS for the object-storage-to-VM-filesystem stuff, it's cool tech.

* https://fly.io/blog/design-and-implementation/ * https://juicefs.com

The Claude Code analytics space is really interesting to me right now as well, this is cool.

I'm coming at it from more of the data infrastructure side (e.g. send all of your logs and metrics to a cheap Iceberg catalog in the cloud so you have a central place to query[1]) but also check out https://github.com/tobilg/ai-observer -- duckdb popping up everything to make this interesting and easy.

[1] https://github.com/smithclay/otlp2pipeline

High-quality digital twins of complex software does not bode well at all for a lot of SaaS companies.

For customers, it makes migrations much easier and less-risky between vendors.

For the vendors themselves, it means you can cheaply and reliably port features your competitors have that you don’t have.

Personally: love the decorator pattern after I got used to it :)

Posted this yesterday as well, but seems like a really nice emerging pythonic way to call out to remote infrastructure (see: Modal[1]).

[1]: https://modal.com/docs/examples/hackernews_alerts#defining-t...

This looks cool, love the approach of a more "pythonic" infra as code -- have also noticed this pattern in Modal [1] and Ray [2] and all for it.

[1] https://modal.com/docs/examples/hackernews_alerts [2] https://docs.ray.io/en/latest/ray-overview/getting-started.h...

edit: forgot to mention StackQL [3], which is somewhat similar but takes a SQL-first vs python-first approach. The IaC space is about to get a lot more interesting.

[3] https://stackql.io/docs

We need more rigorous benchmarks for SRE tasks, which is much easier said that done.

The only other benchmark I've come across is https://sreben.ch/ ... certainly there must be others by now?

Anthropic ships an official plugin to create linters for you based on your Claude Code history or instructions, it’s great. You can vibe code your lint rules per repo.

https://github.com/anthropics/claude-code/tree/main/plugins/...

This is part of a bigger consolidation trend, AI hype or not: which general-purpose data vendor gets to store and query all of your observability and business data?

Snowflake acquired Observe last week, AWS made it easy in December to put logs from Cloudwatch in their managed iceberg catalog, and Azure is doing a bunch of interesting stuff with Fabric.

The line between your data lake/analytics vendor and observability vendor is getting blurry.

This is an emerging pattern that’s surprisingly powerful: thick clients that embed wasm query engines (pglite, duckdb) and do powerful analytics (with or without AI agents writing the queries).

Below are two examples using duckdb under the hood for similar purposes. Like the author, excited for this type of architecture making semi-advanced analytics more attainable if you’re not a data engineer.

- https://pandas-ai.com/ - https://marimo.io/

Agree with the author, will add: duckdb is an extremely compelling choice if you’re a developer and want to embed analytics in your app (which can also run in a web browser with wasm!)

Think this opens up a lot of interesting possibilities like more powerful analytics notebooks like marimo (https://marimo.io/) … and that’s just one example of many.

Kudos to Ben for speaking to one of the elephants in the room in observability: data waste and the impact it has on your bill.

All major vendors have a nice dashboard and sometimes alerts to understand usage (broken down by signal type or tags) ... but there's clearly a need for more advanced analysis which Tero seems to be going after.

Speaking of the elephant in room in observability: why does storing data on a vendor cost so much in the first place? With most new observability startups choosing to store store data in columar formats on cheap object storage, think this is also getting challenged in 2026. The combination of cheap storage with meaningful data could breathe some new life into the space.

Excited to see what Tero builds.

This is great. If you’re skeptical, vibe coding in the go is great because of how async the agentic coding workflows can be. Nothing like fixing a bug in the dentist office.

Lots of different technical solutions for how to do this, including the Claude and ChatGPT mobile apps nowadays. I use Tailscale. Choose what works best for you and enjoy.

Thanks for the PR, blame myself for the issue more than Claude :)

Forgot OTLP log bodies could be arrays/non-strings and was not a test case.

The blog post is not, the rust glue code is.

Been experimenting with OpenTelemetry->Parquet conversion lately for logs, metrics, and traces. Lots of related projects popping up in this area. It's powerful and cheap.

- https://github.com/smithclay/otlp2parquet (shameless plug, based on Clickhouse's Otel schema) - https://github.com/Mooncake-Labs/moonlink (also has OTLP support) - https://github.com/open-telemetry/otel-arrow (official community project under early dev)

My favorite "malleable" example this past week that I recommend: using Gen AI to rewrite, customize, or create dotfiles* for your laptop/desktop.

Defining a bespoke system config was always something that never had the time, desire, or deep familiarity with various config formats to do. With Claude Code or ChatGPT can now generate Wezterm/tmux/neovim/etc config files and get exactly what I want not knowing a single thing about specific config file formats.

* https://dotfiles.github.io

Hey, it's not well known but Anthropic actually publishes their own devcontainer feature layer for this (1). Many different CLI wrappers and tools are starting to imbed it (2) but is a nice DIY/open-source way to sandbox.

1. https://github.com/anthropics/devcontainer-features 2. shameless plug, my own open-source CLI for doing this: https://github.com/smithclay/claudetainer