shcallaway
Tracks along the longest drive on another world
Capturing and replaying real database workloads at Airbnb
Cve.now: A Free Feed of Security Vulnerabilities for Agents
Cve.now: A free CVE feed for agents
Our $8M Seed Round and Open Beta
Sazabi: The AI-native observability platform for fast-moving engineering teams
The Sazabi Manifesto
This isn't the whole picture, but it's a huge part of the picture. IMO, observability shouldn't be so complex that it warrants specialized experience; it should be something that any junior product engineer can do on their own.
> I can definitely imagine having Claude debug an issue faster than I can type and click around dashboards and query UIs. That sounds fun.
Working on it :)
I don't see why the same isn't true for "vibe-fixers" and their data (telemetry).
> We'll have solved the problem when AI detects the problem and submits the bug fix before the engineers wake up.
Working on it :)
Huge fan of historical artifacts like Cantrill's ACM paper
I'll choose this point:
> reliability is still ultimately an incentive problem
This is a fascinating argument and it feels true.
Think about it. Why do companies give a shit about reliability at all? They only care b/c it impacts bottom line. If the app is "reliable enough" such that customers aren't complaining and churning, it makes sense that the company would not make further investments in reliability.
This same logic is true at all levels of the organization, but the signal gets weaker as you go down the chain. A department cares about reliability b/c it impacts the bottom line of the org, but that signal (revenue) is not directly and attributable to the department. This is even more true for a team, or an individual.
I think SLOs are, to some extent, a mechanism that is designed to mitigate this problem; they serve as stronger incentive signals for departments and teams.
My intention wasn't for this post to be a comprehensive historical record. That would have taken many more words & would have put everyone to sleep. My goal was to unpack and analyze _modern observability_ - the version that we are all dealing w/ today.
Good point though!
Observability's Past, Present, and Future
Also, an announcement: I’m starting something new!
Round Two
I don't have much experience w/ "vanilla" LangChain or the LC Python tools, but I've been an avid user of Deployments and the LangGraph TypeScript SDK for like a year now (started using Deployments back when it was still called "LangGraph Platform"). I think I might be one of the oldest users of both...
To be honest, my first impressions were mixed. The Deployments product was very early/new and rough around the edges (bad monorepo support, for example). And the LangGraph TypeScript SDK felt... not super TypeScript-y. (I get the sense they ported over a bunch of abstractions from the Python package).
But the benefits outweighed the costs. In September 2024, Deployments + the LangGraph TypeScript SDK was one of the only ways that you could say FOR SURE that your agent was going to (a) work and (b) run smoothly in production.
(Also, their team worked hard as hell to ramp myself and my teammates up on agents. I was really won over by this and will be a LC stan forever as a result.)
Over the past year, I've seen both products evolve and mature significantly - to the point where all of those initial problems seem to have been addressed.
I still feel that the overhead associated with building and hosting your own agent from scratch is too much for most teams to take on, even at large companies. (In particular, first-class support for streaming is massive; this would be a huge PITA to build in-house.) I'm happy to let LangChain take care of all of this for me.
Overall, I've been very impressed at LC's ability to iterate and absorb feedback - especially when some of it is... not delivered in the kindest way. They're a very humble and hardworking team. Makes me happy to see them winning.