nostrebored
If you weren’t making intentional decisions about cloud vs on prem… why? I worked with many small businesses that did.
This feels like a value-divorced outsourced engineering mindset. You have the power to understand if things are good ideas or working!
You should not write things with AI that you expect people to read
Abstraction is an issue for every AI coding agent. They are painfully useful, but need constant badgering in the right direction.
Almost all of our work is on our framework internally. The latency, guarantees, build time errors, and feedback that can enable vibe coding individual implementations.
I find myself constantly wishing for more framework code; I wish we had built our original framework in something that threw angrily when we abused it more.
I wish our type system were more expressive, I wish our threading story were better, I wish I wish I wish.
It’s never been easier to rework the whole thing, and I still am happiest when our framework is artisanal.
Partially because I can read and validate it actually shows what it claims to show, and partially because I expect people want to know what I’m telling them.
Performance testing has always had problems with isolation, mocking, covariance of services etc. I’ve just spent two weeks driving down latency across our framework, and autoresearch was definitely not a viable path. Most of these loops have these logarithmic, non-step change curves.
Just like with actual research, the real benefits came from me asking why is it like X and not like Y?
The original performance testing framework Claude created to bench our different versions against did not even mock high variance provider calls…
Would you define two apartment units next to each other, with the same floor plan as fungible?
What about the house next door?
For the purposes that people use them for, there is an immense allocation problem in the US. People are sitting on a 5b/5ba and living alone. Bizarre sets of incentives have culminated to make this possible. We have collectively decided that for some reason, it’s completely sane to live in a house that you don’t need. We should look at it the same way as owning a stretch hummer.
Housing is a fungible good. I have friends and family living in squalor because people have decided sentiment is a valid reason to keep people in these four walls instead of those four walls.
I think people’s primary residence should not be seen as assets in the way they are today at all. The only reason they are is substantial government intervention.
I’m definitely on the “just use the clanker” end of the spectrum. But knowing when and is good. Constantly having to argue with people’s reposting of Claude’s idea of what’s wrong has left me burned out. At this point, if I have any idea about what’s going on and someone tells me “Claude says”, I will ignore them.
In this case, it sounds like you could have made a ten line repro that shows the problem. Why not just send that, something human interpretable?
Doing better, I suspect, will be throwing a bigger model underneath and doing something diffusion like (we’ll allocate a whole block to compute rather than just the constrained space) that doesn’t compromise on latency.
System one models do not have an obvious use in voice ai, unless you are choosing from a list of potential utterances (eg embed -> top k -> choose from top k)
Modern stt is baking semantic EOT into the prediction. It makes sense. Conversation has many potential ends, and you have to deal with misprediction of current words that may signal completion, but mel spectrograms contain a ton of information as well.
In your version, we’d have to deal with partial transcriptions and a system that is not natively audio aware. It would also be much more expensive.
People are okay with code that does stuff, however bad it may be.
But at some point it becomes impossible to understand and reconcile, which is something humans struggle with as well. Think of fantasy series that will never culminate, because there is no possible culmination.