brilee
"Holds up"
"not a toy, but not a sprawling project either, and ideally.."
"And that’s the trap"
ctrl F "real" -> 6 usages
ctrl F "genuine" -> 4 usages
As a serial successful field-hopper, I agree that I'm not the right person to be making these estimates.
But the external view is that college courses roughly expect you to do what I'm claiming, in roughly the time investment that I'm claiming -- and undergrads are typically in 4+ classes at a time. So is it that the whole educational system is delusional? (I fully acknowledge it might be so!)
Expertise in the Age of AI
What's in a Codebase?
Running a Minecraft Server and More on a 1960s Univac Computer
What's in a Codebase?
Eight years of wanting, three months of building with AI
What's in a Codebase?
Interviewing for ML/AI Engineers
Taste
I think if you want to give your reader a quick intro to, e.g., what is the Adam optimizer, a simple link to Wikipedia is fine. No need to copy-paste an AI tutorial on Adam into the blog post.
Codebase is 250% AI generated
Taste
Taste
This throughput assumes 100% utilizations. A bunch of things raise the cost at scale:
- There are no on-demand GPUs at this scale. You have to rent them for multi-year contracts. So you have to lock in some number of GPUs for your maximum throughput (or some sufficiently high percentile), not your average throughput. Your peak throughput at west coast business hours is probably 2-3x higher than the throughput at tail hours (east coast morning, west coast evenings) - GPUs are often regionally locked due to data processing issues + latency issues. Thus, it's difficult to utilize these GPUs overnight because Asia doesn't want their data sent to the US and the US doesn't want their data sent to Asia.
These two factors mean that GPU utilization comes in at 10-20%. Now, if you're a massive company that spends a lot of money on training new models, you could conceivably slot in RL inference or model training to happen in these off-peak hours, maximizing utilization.
But for those companies purely specializing in inference, I would _not_ assume that these 90% margins are real. I would guess that even when it seems "10x cheaper", you're only seeing margins of 50%.
Multimodal LLMs Are Blind
Claude as Pipeline Orchestrator
What ChatGPT Study Mode gets right and wrong
Here's one way they could get around their own privacy policy: keep track of what % of Claude-generated code is retained in the codebase over time (as an indicator of how high-quality / bug-free the code was); A/B test variations of Claude Code to see which variations have higher retention percentages.
No usage data is retained, no code is retained, no data is used (other than a single floating point number) and yet they get to improve their product atop your usage patterns.
Here's another idea: use a summarization model to transform your session transcript into a set of bits saying "user was satisfied/dissatisfied with this conversation", "user indicated that claude was doing something dangerous", "user indicated that claude was doing something overly complicated / too simple", "user interrupted claude", "user indicated claude should remember something in CLAUDE.md", etc. etc. and then train on these auxiliary signals, without ever seeing the original code or usage data.
I think it's quite plausible that Anthropic is bleeding out ~100/month on token costs per $20/month user, and even at 80% margin, this is just merely breakeven. Their limited capacity also means that they are _losing_ the opportunity to sell the same capacity at a per-token marginal profit. I think the only plausible endgame here is that Anthropic uses the usage data to RL-finetune Claude Code to the point where it is actually worth a $200/month subscription.
Enjoy the $20/month Claude Pro plan while it lasts; I don't really see it sticking around for more than a year at best.
I think showing the raw reasoning text is not quite the right UI; maybe highlighting the specific text in red and showing a suggested correction would work better?
It's also a little awkward that the conversation is live; I don't really have any breathing room to read the reasoning traces on what mistakes I made / could have done better. I hung up the first time I tried to figure out how to pause.
Tldr: most applications of free energy have capital costs that far outweigh the free energy harvest potential.