jonathanehrlich

What if we trained a smaller KV cache for our documents offline? Using a test-time training recipe we call self-study, we find that this can reduce cache memory on avg 39x (enabling 26x higher tok/s and lower TTFT) while maintaining quality. These smaller KV caches, which we call cartridges, can be trained once and reused for different user requests!

Thanks. Please write more frequently.

Congrats Ryan!!! Huge hire!!

Excited for you guys. You've worked so hard.

Amazing work, Aaron! What an impact! Can't wait to see what you do next....

From what i understand, the FTC reviewed the Instagram acquisition twice before giving their blessing. I wish we all had this kind of power to remake decisions depending on how they turn out. Would be pretty awesome.

@ Aaron - Thanks, as always, for the thoughtful note but I disagree with you on this. Yes, there is risk of over-dilution if you raise boatloads too much out of the gate (including the very real and potentially fatal risk of being undisciplined on spending). But at the end of the day, if a founder figures out the business in that first 12 month window and is still sitting on another 12-24 months of cash, she has so much more power than if she is 4 months to cash out. She can take capital now but exactly on her terms. She can decide to keep pushing and raise in 12-18 months when the business will be in even better shape (and valuations higher, etc). Time becomes her friend, not her enemy. In my opinion, one would be crazy not to take more capital when it's this insanely cheap (valuations for 2 people with a powerpoint have never been higher). As we've seen with so many great companies, p/m often takes more than 8 months. Take more than you need and buy time. It will come out in the wash. I know this may sound self serving given my VC Cloak but having raised my first venture round as a founder in 1998, it comes from a place of experience (and pain). Thanks for listening.