phamilton
One of my favorite things to show just how bare this is in C is to show array access commutativity.
char c = {1,2,3}
c[1] == *(c + 1)
*(c + 1) == *(1 + c)
c[1] == 1[c]
The AWS Aurora white paper was released almost 10 years go. Aurora was announced in 2014 and the paper released in 2017.
(Aurora is built on S3)
A training set of 15 trillion tokens is 10 trillion words.
A penny a word is cheaper than the cheapest beginner freelance writer.
That makes a training set of 10 trillion words cost $100B.
Lots of assumptions there for sure, but we're certainly in the ballpark you are describing.
We had a lot of success with LISTEN/NOTIFY when we paired it with a Rust graphql subscription broker. 10s of thousands of subscriptions, but only 3 or 4 LISTEN connections (one for each host). All changes would be pushed out to all hosts, who would each manage the actual user subscriptions and choose what to actually publish.
This worked super well. In general, moving from hundreds of Ruby or Node hosts to just a few Rust hosts just allows so many simplifications and things that "don't scale" to actually work quite well.
My list:
No `explain (analyze,buffers)`. Instant DDL has some warts (e.g. fk, metadata locks). Query planning bugs (actually... query planning in general is disappointing). Exiting the repl doesn't stop queries. Implicit type casting. Replication lag from large DDL (e.g. creating an index). Lack of two phase DDL (creating constraints NOT VALID and then VALIDATE later). Lack of extensions (e.g. pg_vector). No safe access to inspect buffer cache. AWS Aurora seems to only add shiny new things to Postgres. And more.
Again, none of this is quite enough to migrate off of it for an established system, but certainly enough to avoid it on a new project.
> It only makes backwards compatible deployments easier insofar you're able to do the overhead for splitting the change with less effort then before.
Yes. That reduces the cost of implementing them.
To be clear, I'm not talking about "Split my db migration and my code that depends on the new table". I'm talking about things like "Set up dual writes between an old database schema and a new database schema with a thorough test suite and do shadow reads against both datasets in prod to do differential testing". That's nontrivial engineering effort that would definitely warrant a discussion in the past. Today we just do it. It's fast and lowers risk.
This is exactly how I've felt. I've read some many old papers and books and found great techniques that are even more applicable than ever.
Not sure about hard, but definitely rare and we as an industry are under-skilled in these areas.
We have decades of research and tools for testing and verification of software. Property tests, dependent types, formal verification and proof, etc. The paths have been there, we have just collectively prioritized other things.
It requires an intentional shift in how we design and build. That shift is the harder part.
I mean "We can't build X because our code structure makes that difficult" has an opportunity cost of the value of X.
I don't think the future of dev work is being a bureaucrat. I've done more rigorous engineering the last two years than I did previously. I'm more confident in the things I shop and they were built in a fraction of the time. It's a bright future for software engineering.
The cost of shoring up behavior with tests ahead of a restructure has gone down because of AI.
The cost of implementing a zero downtime migration has gone down because of AI.
A big part of the rust hype has been the low cost of restructuring within an application, even before AI. And now even more so.
The opportunity cost of not being able to safely restructure has gone up substantially.
This is the number one thing I optimize for now: the ability to quickly and safely change significant parts of the code and product.
Each token has to read all the active weights. I think that's around 40B parameters active. At a 4-bit quant that's 20GB. With 100GB/s (replace with whatever your bandwidth is) and you get 5 tokens per second.
1. A robust durability implementation 2. A library of high performance data structure and algorithms
The fact this it's SQL is nice, but those two attributes are what make it great.
For example, I'm implement an in-process event log that I want to be durable. I started simple, but soon saw some edge cases and instead of playing whackamole I just swapped to using sqlite as an ordered kv store that gives me ACID.
Another example: ingesting multiple inter related datasets. Instead of a dozen hash maps in memory, I load them up into sqlite (no persistence) and then slice and dice as I need to.
It's a super useful tool.
How far behind are open models compared to Sonnet?
It may be that the absolute SOTA models are way ahead of open models, but the gap in the mid tier really does feel like it's compressing. I'd love to see empirical data about it though.
It's usable. I set it loose on the postgres codebase, told it to find or build a performance benchmark for the bloom filter index and then identify a performance improvement. It took a long time (overnight), but eventually presented an alternate hashing algorithm with experimental data on false positive rate, insertion speed and lookup speed. There wasn't a clear winner, but it was a reasonable find with rigorous data.
I've been leaning in more on e2e test suites. They are slow, brittle and inefficient. But that's almost a feature. I can step away and come back an hour later, and use that time to think about bigger problems.
Since it's a procedural language, you can't do things like create a new index implementation or something else super low level. But there's still a lot you _can_ do. Like implement a custom comparator for a custom type and then use that type in a btree index.
An empty public company with $70M in financing to enter a hyped market was valued at $115M. The stated intent is to spend their money on a CapEx item with a fairly high demand and resale value (GPUs) in a sector that has a pretty simple playbook.
The 580% bump is a fun headline, but "startup secures $50M in funding at a 5.8x valuation bump" isn't unheard of.
Have I invested? No. Is this a ridiculously funny narrative and story? Absolutely. Is it the most ridiculous valuation I've seen? No.
Namely, all remote access (including serving http) must managed by a major player big enough to be part of private disclosure (e.g. Project Glasswing).
That doesn't mean we have to use AWS et al for everything, but some sort of zero trust solution actively maintained by one of them seems like the right path. For example, I've started running on Hetzner with Cloudflare Tunnels.
Anyone else doing something similar?
The shoe business was sold, a shell of a public company was left, and it essentially acquired a brand new company focused on AI.
A bit of 220 grit sandpaper and all the sharp edges are smooth and it actually looks pretty cool. I was grimacing at first but now I like the feel.
So yeah, Apple probably does pay less. But the market has enough demand that suppliers do say no.
It sounds like there was at least a deliberate attempt to improve it.