gaogao

Do you have any personal stake in the enterprise?

Frequent minor version bumps are pretty common these days. Opus 4.7 -> 4.8 was 42 days.

Doesn't |& work with bash?

Nah, the articles are non-contradicting. That article focuses on how the spend mostly goes to imports, which decreases GDP. This one focuses on the effects on unemployment. It's very plausible that a decrease in interest rates right now would lead to more imports and AI spending, not increased employment.

Yes, though often the easiest way to replicate it in regular programming languages is to translate that language to Lean or another ITM, though auto-active like Verus is used for Rust pretty successfully.

Python and C though have enough nasal demons and undefined behavior that it's a huge pain to verify things about them, since some random other thread can drive by and modify memory in another thread.

The future is now. The recent couple of Lean releases have leaned more on software verification, I believe in part due to Amazon hiring a number of the core devs.

Yeah, I pointed a custom thing and Claude at #6, and it's solved it in Lean besides needing to axiomize one theorem not in mathlib. Only about four of the problems have enough foundations formalized in mathlib though for this approach.

The compiler for CUDA Tile being Blackwell only is a baffling decision. I wanted to try it out, but it's only really easy to grab H100s quickly right now. I guess maybe I'll try it out on my 5070 Ti after traveling, but am more likely to stick to an IR that targets multiple platforms, since they couldn't be bothered.

Yup, I've already spent like $20k using Claude to verify things, so like there's probably some room for cost cutting.

Yeah, Rust has been pretty good for formal verification so far. Hoare spec contracts I think are the way forward, especially since they fairly naturally flow from unittests. I've been using Hax to pretty good effect so far. I'm generally suspect that advances in Lean proof solving by dedicated models are that useful for program verification, compared to generalist models, though it could help lower costs a good bit.

Topical to my interests, I used Claude Code the other day for formally verifying some matrix multiplication in Rust. Writing the spec wasn't too hard actually, done as post-conditions in code, as proving equivalence to a simpler version of the code, such as for optimization, is pretty straight forward. Maybe I should write up a proper post on it.

> Season 8: In this season, they introduced gender fluidity. Whilst an interesting problem on its own, this would have wreaked havoc on my model.

Well I guess free money except for that one. In that one, one of the contestants, Danny, did the math for optimizing their remaining Truth Booths and Match Ups to get it down to a 50/50 shot.

Pathways, I understand, is more so these days just the name for their training orchestrator for doing distributed JAX stuff - https://github.com/google/pathways-job

The gaps that led to this was, I think, part of why the CISO got replaced - https://www.thestack.technology/anthropic-new-ciso-claude-cy...

> In 2007, the scientist, who once worked at the University of Cambridge's Cavendish Laboratory, told the Times newspaper that he was "inherently gloomy about the prospect of Africa" because "all our social policies are based on the fact that their intelligence is the same as ours - whereas all the testing says not really".

> While his hope was that everybody was equal, he added, "people who have to deal with black employees find this is not true".

Yeah, pretty racist

If asked non-directly, it still currently answers it - https://www.kimi.com/share/19a5ab4a-e732-8b8b-8000-00008499c...

Ah so, this is pretty possible, as it's a flavor of text adventure puzzle that usually shows up every year or two for Mystery Hunt. I agree that visually allows this to a greater degree, but it's exceedingly doable to have such a layered game just in text.

Oh also good talk at PTC yesterday! I had meant to ask you more about the formal memory model, but the other post talk questions ended up being really interesting too.

Yup, hyperreactor, one of the new crates that's part of it, does some particularly interesting things for efficient parallel distributed channels.

> It's a pity they don't do a complete rewrite with a functional language as the driver.

It's open source, so seeing such an extension would be quite cool. There's much that could be done with native Rust actors and code that get maybe at what you want, but nothing precludes mixing PyTorch and other backends.

For example, you could wrap a C++ inference engine as part of one of the actors generating data for other actors doing distributed training.

> As far as things that might be a performance loss here, one thing I'm wondering is if custom kernels are supported

Yeah, you might end up needing some changes to remote worker initialization, but you can generally bake in whatever kernels and other system code you need.

Nah, focusing on a different controller paradigm. Jax is focused on multi-controller SPMD, while this is focused on a single-controller setup. Both have their place, with single-controller being generally easier to reason about, and multi-controller more optimal for certain dataflows. There's also some interesting mixes of the two control paradigms.

Yup, there's stuff like https://pytorch.org/blog/introducing-torchforge/ on top of it now

Do you think it might be easier to target cuTile instead of PTX? (Probably not, since it has a less formalized model?)

Yeah, I don't think surrealism or constructed is good in the early data mix, but as part of mid or post-training seems generally reasonable. But also, this is one of those cases where anthropomorphizing the model probably doesn't work, since a major negative effect of Cocomelon is kids only wanting to watch Cocomelon, while for large model training, it doesn't have much choice in the training data distribution.

Brain rot texts seems reasonably harmful, but brain rot videos are often surreal and semantically dense in a way that probably improves performance (such as discussed on this German brain rot analysis https://www.youtube.com/watch?v=-mJENuEN_rs&t=37s). For example, Švankmajer is basically proto-brainrot, but is also the sort of thing you'd watch in a museum and think about.

Basically, I think the brain rot aspect might be a bit of terminology distraction here, when it seems what they're measuring is whether it's a puff piece or dense.

Iffy by the headline, but the every-other-week model doesn't sound absolutely the worst. I don't have a great read on if it's a bad idea or not though.

Yeah, it's under "Non-SI units accepted for use with SI", so more ISO than SI, but has standards for how it's used with SI units, esp since M has meaning there

Yeah, the SI symbol for minutes is min, if you're going to abbreviate it in a technical context. Super funky using M.

> I wonder if we'll ever use AI to write code in dependently typed languages

Yeah, I write code in Lean with AI pretty frequently lately